Imaging element, imaging apparatus, image data processing method, and program that performs imaging in a first frame rate and outputs data in a second frame rate

ABSTRACT

An imaging element includes: a memory that stores captured image data obtained by imaging a subject at a first frame rate; an image processing circuit that performs processing on the captured image data; and an output circuit that outputs output image data obtained by performing the processing on the captured image data to an exterior of the imaging element at a second frame rate, wherein the image processing circuit performs cut-out processing with respect to one frame of the captured image data, the cut-out processing including cutting out partial image data indicating an image of a part of the subject in the captured image data from a designated address in the memory, the output image data includes image data based on the partial image data that is cut out from the captured image data, and the first frame rate is a frame rate higher than the second frame rate.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation application of InternationalApplication No. PCT/JP2019/025646, filed on Jun. 27, 2019, which isincorporated herein by reference in its entirety. Further, thisapplication claims priority from Japanese Patent Application No.2018-163996, filed on Aug. 31, 2018, the disclosure of which isincorporated by reference herein in their entirety.

BACKGROUND Technical Field

The technology of the present disclosure relates to an imaging element,an imaging apparatus, an image data processing method, and anon-transitory program storage medium.

Related Art

JP2016-225970A discloses an imaging apparatus including an imagingelement, a signal processing portion that performs predetermined signalprocessing on image data output from the imaging element, a displayportion that displays an image, and a control portion that controls eachof the imaging element, the signal processing portion, and the displayportion.

The imaging element disclosed in JP2016-225970A comprises a firstsemiconductor substrate and a second semiconductor substrate that arelaminated with each other and are electrically directly connected. Animaging unit that receives and photoelectrically converts an incidenceray, and an AD conversion unit that converts an analog image signaloutput from the imaging unit into digital image data are disposed in thefirst semiconductor substrate. A storage unit that stores the digitalimage data of one frame converted by the AD conversion unit, and aprocessing unit that performs resizing processing on the digital imagedata stored in the storage unit are disposed in the second semiconductorsubstrate. In the imaging element disclosed in JP2016-225970A, theprocessing unit cuts out image data of a predetermined region from thedigital image data of one frame before being stored in the storage unitand stores the cut-out image data in the storage unit.

JP2018-007210A discloses an imaging apparatus including an imaging unitand a control unit. The imaging unit included in the imaging apparatusdisclosed in JP2018-007210A comprises a plurality of pixels and outputsa first image signal acquired by a first pixel group of the plurality ofpixels and a second image signal acquired by a second image group of theplurality of pixels. The control unit included in the imaging apparatusdisclosed in JP2018-007210A performs a first control and a secondcontrol. The first control is performing processing based on the secondimage signal and displaying a first image corresponding to the firstimage signal on a display screen. The second control is displaying asecond image obtained by combining the first image signal with thesecond image signal on a display screen.

SUMMARY

One embodiment of the present invention provides an imaging element, animaging apparatus, an image data processing method, and a non-transitorystorage medium storing a program capable of reducing power consumption,compared to a case of cutting out partial image data from image datathat is output to the outside of an imaging element without passingthrough a storage portion (i.e., memory), and outputting the partialimage data.

A first aspect according to the technology of the present disclosure isan imaging element comprising a memory that stores captured image dataobtained by imaging a subject at a first frame rate, the memory beingincorporated in the imaging element; an image processing circuit thatperforms processing on the captured image data, the image processingcircuit being incorporated in the imaging element; and an output circuitthat outputs output image data obtained by performing the processing onthe captured image data to an exterior of the imaging element at asecond frame rate, the output circuit being incorporated in the imagingelement, wherein the image processing circuit performs cut-outprocessing with respect to one frame of the captured image data, thecut-out processing including cutting out partial image data indicatingan image of a part of the subject in the captured image data from adesignated address in the memory, the output image data includes imagedata based on the partial image data that is cut out from the capturedimage data as a result of performing the cut-out processing by the imageprocessing circuit, and the first frame rate is a frame rate higher thanthe second frame rate.

Accordingly, the imaging element of the first aspect according to thetechnology of the present disclosure can reduce power consumptioncompared to a case of cutting out the partial image data from image dataoutput to the outside of the imaging element without passing through thestorage portion, and outputting the partial image data.

A second aspect according to the technology of the present disclosure isthe imaging element according to the first aspect, in which the cut-outprocessing is processing of cutting out the partial image data from thecaptured image data stored in the memory by random access to the memory.

Accordingly, the imaging element of the second aspect according to thetechnology of the present disclosure can quickly cut out the partialimage data from the designated address compared to a case where randomaccess to the memory is not available.

A third aspect according to the technology of the present disclosure isthe imaging element according to the first aspect or the second aspect,wherein the memory is capable of storing the captured image data of aplurality of frames, the cut-out processing includes processing ofcutting out the partial image data of the plurality of frames from thecaptured image data stored in the memory, and the output portionoutputs, as the output image data, image data that is obtained bycombining the partial image data of the plurality of frames cut out as aresult of performing the cut-out processing by the image processingcircuit.

Accordingly, the imaging element of the third aspect according to thetechnology of the present disclosure can reduce power consumptioncompared to a case of outputting the captured image data to the outsideof the imaging element, cutting out the partial image data of theplurality of frames from the captured image data outside the imagingelement, and combining the partial image data of the plurality offrames.

A fourth aspect according to the technology of the present disclosure isthe imaging element according to the third aspect, in which the partialimage data is at least one of a plurality of pieces of divided regionimage data corresponding to a plurality of divided regions obtained bydividing the part.

Accordingly, the imaging element of the fourth aspect according to thetechnology of the present disclosure can reduce power consumptioncompared to a case of cutting out the partial image data from thecaptured image data of each of the plurality of frames and combining thecut-out partial image data of the plurality of frames.

A fifth aspect according to the technology of the present disclosure isthe imaging element according to the fourth aspect, in which theplurality of pieces of divided region image data have a relationship inwhich mutually different pixels are thinned out in units of lines.

Accordingly, the imaging element of the fifth aspect according to thetechnology of the present disclosure can obtain a subject image of highreproducibility compared to a case of thinning out pixels at the sameposition in units of lines.

A sixth aspect according to the technology of the present disclosure isthe imaging element according to the fourth aspect or the fifth aspect,in which the output image data includes combined data obtained bycombining the plurality of pieces of divided region image data.

Accordingly, the imaging element of the sixth aspect according to thetechnology of the present disclosure can provide an image of high imagequality compared to a case of outputting the partial image data of asingle frame.

A seventh aspect according to the technology of the present disclosureis the imaging element according to the sixth aspect, in which theoutput circuit selectively outputs, as the partial image data, dividedregion image data of a part of the plurality of pieces of divided regionimage data or the combined data in accordance with a provided condition.

Accordingly, the imaging element of the seventh aspect according to thetechnology of the present disclosure can reduce power consumptioncompared to a case of outputting the combined data to the outside of theimaging element from the imaging element at all times.

An eighth aspect according to the technology of the present disclosureis the imaging element according to any one of the first aspect to theseventh aspect, wherein the subject is imaged by a photoelectricconversion element, in a case in which an area of the photoelectricconversion element corresponding to the part is designated bydesignating the part of the subject, the image processing circuit causesthe area of the photoelectric conversion element to perform imaging, andthe image processing circuit acquires partial image correspondence datacorresponding to the partial image data at the first frame rate byperforming imaging by the area of the photoelectric conversion element,and outputs image data based on the acquired partial imagecorrespondence data to the exterior of the imaging element at the secondframe rate.

Accordingly, the imaging element of the eighth aspect according to thetechnology of the present disclosure can reduce power consumptioncompared to a case of imaging the subject using all regions (i.e., theentire area) of the photoelectric conversion element at all times.

A ninth aspect according to the technology of the present disclosure isthe imaging element according to any one of the first aspect to theeighth aspect, in which the part includes a plurality of regions in thesubject, and the partial image data is a plurality of pieces of regionimage data indicating the plurality of regions in the captured imagedata.

Accordingly, the imaging element of the ninth aspect according to thetechnology of the present disclosure can reduce power consumption evenin a case where a cut-out target is present across a plurality ofregions of the subject.

A tenth aspect according to the technology of the present disclosure isthe imaging element according to any one of the first aspect to theninth aspect, in which the output image data further includes wide rangeimage data that indicates an image of a range of the subject wider thanthe part of the subject, and resolution of the wide range image data islower than resolution of the partial image data.

Accordingly, the imaging element of the tenth aspect according to thetechnology of the present disclosure can reduce power consumptioncompared to a case where the resolution of the wide range image data isthe same as the resolution of the partial image data.

An eleventh aspect according to the technology of the present disclosureis the imaging element according to the tenth aspect, in which the widerange image data is image data indicating an image of an entirety of thesubject.

Accordingly, the imaging element of the eleventh aspect according to thetechnology of the present disclosure can reduce power consumptioncompared to a case where resolution of the image data indicating theimage of the entirety of the subject is the same as the resolution ofthe partial image data.

A twelfth aspect according to the technology of the present disclosureis the imaging element according to any one of the first aspect to theeleventh aspect, in which the output image data further includesfocusing control image data indicating an image of a range that is arange of the subject wider than the part of the subject and that is apredetermined range as a range required for a focusing control of animaging apparatus including the imaging element.

Accordingly, even in a case where the focusing control is performedusing the focusing control image data, the imaging element of thetwelfth aspect according to the technology of the present disclosure canreduce power consumption even compared to a case of generating thefocusing control image data outside the imaging element.

A thirteenth aspect according to the technology of the presentdisclosure is the imaging element according to any one of the firstaspect to the twelfth aspect, in which the imaging element is a stackedimaging element that includes a photoelectric conversion element, and inwhich the photoelectric conversion element is stacked on the memory.

Accordingly, the imaging element of the thirteenth aspect according tothe technology of the present disclosure can output image data subjectedto image processing in the imaging element to the outside of the imagingelement.

A fourteenth aspect according to the technology of the presentdisclosure is an imaging apparatus comprising the imaging elementaccording to any one of the first aspect to the thirteenth aspect of thetechnology of the present disclosure, and a display processor that isconfigured to perform a control for displaying an image based on theoutput image data output by the image processing circuit on a display inan enlarged manner.

Accordingly, the imaging apparatus of the fourteenth aspect according tothe technology of the present disclosure can reduce power consumptioncompared to a case of cutting out the partial image data from image dataoutput to the outside of the imaging element without passing through thestorage portion, and outputting the partial image data.

A fifteenth aspect according to the technology of the present disclosureis an image data processing method of an imaging element in which amemory, an image processing circuit, and an output circuit areincorporated, the image data processing method comprising: storing, bythe memory, captured image data obtained by imaging a subject at a firstframe rate; performing, by the image processing circuit, processing onthe captured image data; outputting, by the output circuit, output imagedata obtained by performing the processing on the captured image data toan exterior of the imaging element at a second frame rate; andperforming, by the image processing circuit, cut-out processing withrespect to one frame of the captured image data, the cut-out processingincluding cutting out partial image data indicating an image of a partof the subject in the captured image data from a designated address inthe memory, wherein the output image data includes image data based onthe partial image data that is cut out from the captured image data as aresult of performing the cut-out processing by the image processingcircuit, and the first frame rate is a frame rate higher than the secondframe rate.

Accordingly, the image data processing method of the fifteenth aspectaccording to the technology of the present disclosure can reduce powerconsumption compared to a case of cutting out the partial image datafrom image data output to the outside of the imaging element withoutpassing through the storage portion, and outputting the partial imagedata.

A sixteenth aspect according to the technology of the present disclosureis a non-transitory storage medium storing a program that causes acomputer to function as an image processing circuit and an outputportion included in an imaging element, and to perform an image dataprocessing, the imaging element incorporating a memory, the imageprocessing circuit, and the output circuit, the image data processingcomprising: storing, by the memory, captured image data obtained byimaging a subject at a first frame rate; performing, by the imageprocessing circuit, processing on the captured image data; outputting,by the output circuit, output image data obtained by performing theprocessing on the captured image data to an exterior of the imagingelement at a second frame rate; and performing, by the image processingcircuit, cut-out processing with respect to one frame of the capturedimage data, the cut-out processing including cutting out partial imagedata indicating an image of a part of the subject in the captured imagedata from a designated address in the memory, wherein the output imagedata includes image data based on the partial image data that is cut outfrom the captured image data as a result of performing the cut-outprocessing by the image processing circuit, and the first frame rate isa frame rate higher than the second frame rate.

Accordingly, the program of the sixteenth aspect according to thetechnology of the present disclosure can reduce power consumptioncompared to a case of cutting out the partial image data from image dataoutput to the outside of the imaging element without passing through thestorage portion, and outputting the partial image data.

A seventeenth aspect according to the technology of the presentdisclosure is an imaging element comprising a memory that storescaptured image data obtained by imaging a subject at a first frame rateand is incorporated in the imaging element, a processor that performsprocessing on the captured image data, outputs output image dataobtained by performing the processing on the captured image data to anoutside of the imaging element at a second frame rate, and isincorporated in the imaging element, in which the processor performscut-out processing of cutting out partial image data indicating an imageof a part of the subject in the captured image data from a designatedaddress in the memory, the output image data includes image data basedon the partial image data that is cut out from the captured image databy performing the cut-out processing by the processor, and the firstframe rate is a frame rate higher than the second frame rate.

According to one embodiment of the present invention, an effect of beingable to reduce power consumption compared to a case of cutting out thepartial image data from the image data output to the outside of theimaging element without passing through the storage portion, andoutputting the partial image data is achieved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a perspective view illustrating one example of an exterior ofan imaging apparatus that is an interchangeable lens camera according tofirst to fourth embodiments.

FIG. 2 is a rear view illustrating a rear surface side of the imagingapparatus according to the first to fourth embodiments.

FIG. 3 is a block diagram illustrating one example of a hardwareconfiguration of the imaging apparatus according to the first to fourthembodiments.

FIG. 4 is a schematic configuration diagram illustrating one example ofa configuration of a hybrid finder included in the imaging apparatusaccording to the first to fourth embodiments.

FIG. 5 is a schematic configuration diagram illustrating one example ofa schematic configuration of an imaging element included in the imagingapparatus according to the first to fourth embodiments.

FIG. 6 is a block diagram illustrating one example of a mainconfiguration of the imaging element included in the imaging apparatusaccording to the first to fourth embodiments.

FIG. 7 is a flowchart illustrating one example of a flow of image datageneration processing according to the first embodiment.

FIG. 8 is a flowchart illustrating one example of a flow of image dataoutput processing according to the first to fourth embodiments.

FIG. 9 is a flowchart illustrating one example of a flow of displaycontrol processing according to the first to third embodiments.

FIG. 10 is a conceptual diagram illustrating one example of a schematicconfiguration of captured image data and combined data of a first frameto a third frame.

FIGS. 11A and 11B are state transition diagrams illustrating one exampleof a flow of data of the imaging apparatus. FIG. 11A is a statetransition diagram illustrating one example of a flow of data of animaging apparatus according to a technology in the related art. FIG. 11Bis a state transition diagram illustrating one example of a flow of dataof an imaging apparatus according to an embodiment of the technology ofthe present disclosure.

FIG. 12 is a state transition diagram illustrating one example of a flowof data of the imaging apparatus according to the embodiment of thetechnology of the present disclosure.

FIGS. 13A and 13B are state transition diagrams illustrating one exampleof a flow of processing of the imaging apparatus. FIG. 13A is a statetransition diagram illustrating one example of a flow of processing ofthe imaging apparatus according to the technology in the related art.FIG. 13B is a state transition diagram illustrating one example of aflow of processing of the imaging apparatus according to the embodimentof the technology of the present disclosure.

FIG. 14 is a conceptual diagram illustrating a state example in which animage showing a face region of a person who is a subject is displayed inan enlarged manner after the face region is detected.

FIG. 15 is a conceptual diagram illustrating a first modificationexample of the schematic configuration of the captured image data andthe combined data of the first frame to the third frame.

FIG. 16 is a conceptual diagram illustrating a second modificationexample of the schematic configuration of the captured image data andthe combined data of the first frame to the third frame.

FIG. 17 is a descriptive diagram for describing a difference betweenthinned output image data and non-thinned output image data.

FIGS. 18A and 18B are descriptive diagrams for describing an exposuremethod of the imaging apparatus according to the first to fourthembodiments. FIG. 18A is a descriptive diagram for describing a firstexposure method of the imaging apparatus according to the first tofourth embodiments. FIG. 18B is a descriptive diagram for describing asecond exposure method of the imaging apparatus according to the firstto fourth embodiments.

FIG. 19 is a state transition diagram illustrating one example of a flowof data of the imaging apparatus according to the embodiment of thetechnology of the present disclosure.

FIG. 20 is a state transition diagram illustrating one example of a flowof data of the imaging apparatus according to the embodiment of thetechnology of the present disclosure.

FIG. 21 is a flowchart illustrating one example of a flow of image datageneration processing according to the second embodiment.

FIG. 22 is a flowchart illustrating one example of a flow of singleregion cut-out processing according to the second embodiment.

FIG. 23 is a flowchart illustrating one example of a flow of pluralregion cut-out processing according to the second embodiment.

FIG. 24 is a state transition diagram illustrating one example of a flowof data of the imaging apparatus according to the embodiment of thetechnology of the present disclosure.

FIG. 25 is a state transition diagram illustrating one example of a flowof data of the imaging apparatus according to the embodiment of thetechnology of the present disclosure.

FIG. 26 is a flowchart illustrating one example of a flow of image datageneration processing according to the third embodiment.

FIG. 27 is a state transition diagram illustrating one example of a flowof data of the imaging apparatus according to the embodiment of thetechnology of the present disclosure.

FIG. 28 is a flowchart illustrating one example of a flow of outputimage data processing according to the third embodiment.

FIG. 29 is a flowchart illustrating one example of a flow of imagingcontrol processing according to the fourth embodiment.

FIG. 30 is a flowchart illustrating one example of a flow of image datageneration processing according to the fourth embodiment.

FIG. 31 is a conceptual diagram illustrating one example of a statewhere a program according to the embodiment is installed on the imagingelement from a storage medium storing the program according to theembodiment.

FIG. 32 is a block diagram illustrating one example of a schematicconfiguration of a smart device in which the imaging element accordingto the first to fourth embodiments is incorporated.

DETAILED DESCRIPTION

Hereinafter, one example of an embodiment of an imaging apparatusaccording to the embodiment of the technology of the present disclosurewill be described in accordance with the appended drawings.

First Embodiment

For example, as illustrated in FIG. 1 , an imaging apparatus 10 is aninterchangeable lens camera. The imaging apparatus 10 is a digitalcamera that includes an imaging apparatus main body 12 and aninterchangeable lens 14 interchangeably mounted on the imaging apparatusmain body 12, and does not include a reflex mirror. The interchangeablelens 14 includes an imaging lens 18 including a focus lens 16 that ismovable in an optical axis direction by a manual operation.

A hybrid finder (registered trademark) 21 is disposed in the imagingapparatus main body 12. For example, the hybrid finder 21 here refers toa finder in which an optical view finder (hereinafter, referred to asthe OVF) and an electronic view finder (hereinafter, referred to as theEVF) are selectively used. The abbreviation OVF stands for “Optical ViewFinder”. The abbreviation EVF stands for “Electronic View Finder”.

The interchangeable lens 14 is interchangeably mounted on the imagingapparatus main body 12. A focus ring 22 that is used at a time of amanual focus mode is disposed in a lens barrel of the interchangeablelens 14. The focus lens 16 moves in the optical axis direction inaccordance with a manual rotation operation of the focus ring 22, and animage of subject light is formed on an imaging element 20 (refer to FIG.3 ), described later, at a focal position corresponding to a subjectdistance.

A finder window 24 of the OVF included in the hybrid finder 21 isdisposed on a front surface of the imaging apparatus main body 12. Inaddition, a finder switching lever (finder switching portion) 23 isdisposed on the front surface of the imaging apparatus main body 12. Ina case where the finder switching lever 23 is rotationally moved in adirection of an arrow SW, switching is performed between an opticalimage that is visually recognizable by the OVF, and an electronic image(live view image) that is visually recognizable by the EVF.

An optical axis L2 of the OVF is an optical axis different from anoptical axis L1 of the interchangeable lens 14. A release button 25 anda dial 28 for setting such as a mode of an imaging system and a mode ofa playback system are disposed on an upper surface of the imagingapparatus main body 12.

The release button 25 functions as an imaging preparation instructionportion and an imaging instruction portion, and a push operation of twostages of an imaging preparation instruction state and an imaginginstruction state can be detected. For example, the imaging preparationinstruction state refers to a state where a push is performed to anintermediate position (half push position) from a standby position, andthe imaging instruction state refers to a state where a push isperformed to a final push position (full push position) beyond theintermediate position. Hereinafter, the “state where a push is performedto the half push position from the standby position” will be referred toas a “half push state”, and the “state where a push is performed to thefull push position from the standby position” will be referred to as a“full push state”.

In the imaging apparatus 10 according to a first embodiment, an imagingmode and a playback mode are selectively set as an operation mode inaccordance with an instruction from a user. In the imaging mode, themanual focus mode and an auto focus mode are selectively set inaccordance with an instruction from the user. In the auto focus mode, animaging condition is adjusted by causing the release button 25 to enterthe half push state, and then, exposure is performed in a case where thefull push state is subsequently set. That is, after an exposure state isset by an automatic exposure (AE) function by causing the release button25 to enter the half push state, a focusing control is performed by anauto-focus (AF) function, and imaging is performed in a case where therelease button 25 is caused to enter the full push state.

For example, as illustrated in FIG. 2 , a touch panel display 30, across key 32, a menu key 34, an instruction button 36, and a findereyepiece portion 38 are disposed on a rear surface of the imagingapparatus main body 12.

The touch panel display 30 comprises a liquid crystal display(hereinafter, referred to as a “first display”) 40 and a touch panel 42(refer to FIG. 3 ).

The first display 40 displays an image, a text information, and thelike. The first display 40 is used for displaying a live view image(live preview image) that is one example of a consecutive frame imageobtained by imaging in consecutive frames at a time of the imaging mode.The first display 40 is also used for displaying a still picture that isone example of a single frame image obtained by imaging in a singleframe in a case where an instruction to image a still picture isprovided. Furthermore, the first display 40 is used for displaying aplayback image at a time of the playback mode and/or displaying a menuscreen or the like.

The touch panel 42 is a transmissive touch panel and is overlaid on asurface of a display region of the first display 40. The touch panel 42detects a contact of an instruction object such as a finger or a styluspen. The touch panel 42 outputs detection result information indicatinga detection result (presence or absence of the contact of theinstruction object with the touch panel 42) to a predetermined outputdestination (for example, a CPU 52 (refer to FIG. 3 ) described later)in a predetermined cycle (for example, 100 milliseconds). In a casewhere the touch panel 42 detects the contact of the instruction object,the detection result information includes two-dimensional coordinates(hereinafter, referred to as the “coordinates”) for specifying a contactposition of the instruction object on the touch panel 42. In a casewhere the touch panel 42 does not detect the contact of the instructionobject, the detection result information does not include thecoordinates.

The cross key 32 has a function as a multifunction key that outputsvarious instruction signals for selecting one or a plurality of menus,zooming, and/or frame advance or the like. The menu key 34 is anoperation key that has both of a function as a menu button for providingan instruction to display one or a plurality of menus on a screen of thefirst display 40 and a function as an instruction button for providingan instruction for confirmation, execution, and the like of a selectedcontent. The instruction button 36 is operated in a case of deleting adesired target such as a selected item, canceling a designated content,and returning to an immediately previous operation state.

The imaging apparatus 10 has a still picture imaging mode and a motionpicture imaging mode as an operation mode of the imaging system. Thestill picture imaging mode is an operation mode in which a still pictureobtained by imaging a subject by the imaging apparatus 10 is recorded,and the motion picture imaging mode is an operation mode in which amotion picture obtained by imaging the subject by the imaging apparatus10 is recorded.

For example, as illustrated in FIG. 3 , the imaging apparatus 10includes a mount 46 (refer to FIG. 1 ) comprised in the imagingapparatus main body 12, and a mount 44 on an interchangeable lens 14side corresponding to the mount 46. The interchangeable lens 14 isinterchangeably mounted on the imaging apparatus main body 12 by joiningthe mount 44 to the mount 46.

The imaging lens 18 includes a sliding mechanism 48 and a motor 50. Thesliding mechanism 48 moves the focus lens 16 along the optical axis L1by operating the focus ring 22. The focus lens 16 is slidably attachedto the sliding mechanism 48 along the optical axis L1. The motor 50 isconnected to the sliding mechanism 48, and the sliding mechanism 48slides the focus lens 16 along the optical axis L1 by receiving motivepower of the motor 50.

The motor 50 is connected to the imaging apparatus main body 12 throughthe mounts 44 and 46, and driving thereof is controlled in accordancewith a command from the imaging apparatus main body 12. In the firstembodiment, a stepping motor is applied as one example of the motor 50.Accordingly, the motor 50 operates in synchronization with pulse powerin accordance with a command from the imaging apparatus main body 12.While an example in which the motor 50 is disposed in the imaging lens18 is illustrated in the example illustrated in FIG. 3 , the technologyof the present disclosure is not limited thereto, and the motor 50 maybe disposed in the imaging apparatus main body 12.

The imaging apparatus 10 is a digital camera that records the stillpicture and the motion picture obtained by imaging the subject. Theimaging apparatus main body 12 comprises an operation portion 54, anexternal interface (I/F) 63, and a rear stage circuit 90. The rear stagecircuit 90 is a circuit on a side of receiving data transmitted from theimaging element 20. In the first embodiment, an integrated circuit (IC)is employed as the rear stage circuit 90. Large-scale integration (LSI)is illustrated as one example of the IC.

The rear stage circuit 90 includes the central processing unit (CPU) 52,an I/F 56, a primary storage portion 58, a secondary storage portion 60,an image processing portion 62, a first display control portion 64, asecond display control portion 66, a position detection portion 70, anda device control portion 74. A single CPU is illustrated as the CPU 52in the first embodiment. However, the technology of the presentdisclosure is not limited thereto, and a plurality of CPUs may beemployed instead of the CPU 52. That is, various types of processingexecuted by the CPU 52 may be executed by one processor or a pluralityof physically separated processors.

In the first embodiment, each of the image processing portion 62, thefirst display control portion 64, the second display control portion 66,the position detection portion 70, and the device control portion 74 isimplemented by an application specific integrated circuit (ASIC).However, the technology of the present disclosure is not limitedthereto. For example, instead of the ASIC, at least one of aprogrammable logic device (PLD) or a field-programmable gate array(FPGA) may be employed. Alternatively, at least one of the ASIC, thePLD, or the FPGA may be employed. Alternatively, a computer including aCPU, a read only memory (ROM), and a random access memory (RAM) may beemployed. The CPU may be a single CPU or a plurality of CPUs. Inaddition, at least one of the image processing portion 62, the firstdisplay control portion 64, the second display control portion 66, theposition detection portion 70, or the device control portion 74 may beimplemented by a combination of a hardware configuration and a softwareconfiguration.

The CPU 52, the I/F 56, the primary storage portion 58, the secondarystorage portion 60, the image processing portion 62, the first displaycontrol portion 64, the second display control portion 66, the operationportion 54, the external I/F 63, and the touch panel 42 are connected toeach other through a bus 68.

The CPU 52 controls the entire imaging apparatus 10. In the imagingapparatus 10 according to the first embodiment, at a time of the autofocus mode, the CPU 52 performs the focusing control by controllingdriving of the motor 50 such that a contrast value of the image obtainedby imaging is maximized. In addition, at the time of the auto focusmode, the CPU 52 calculates AE information that is a physical quantityindicating brightness of the image obtained by imaging. In a case wherethe release button 25 is caused to enter the half push state, the CPU 52derives a shutter speed and an F number corresponding to the brightnessof the image indicated by the AE information. The exposure state is setby controlling each related portion to achieve the derived shutter speedand the F number.

The primary storage portion 58 means a volatile memory and refers to,for example, a RAM. The secondary storage portion 60 means anon-volatile memory and refers to, for example, a flash memory or a harddisk drive (HDD).

The operation portion 54 is a user interface that is operated by theuser in a case of providing various instructions to the rear stagecircuit 90. The operation portion 54 includes the release button 25, thedial 28, the finder switching lever 23, the cross key 32, the menu key34, and the instruction button 36. Various instructions received by theoperation portion 54 are output to the CPU 52 as an operation signal,and the CPU 52 executes processing corresponding to the operation signalinput from the operation portion 54.

The position detection portion 70 is connected to the CPU 52. Theposition detection portion 70 is connected to the focus ring 22 throughthe mounts 44 and 46, detects a rotation angle of the focus ring 22, andoutputs rotation angle information indicating the rotation angle that isa detection result to the CPU 52. The CPU 52 executes processingcorresponding to the rotation angle information input from the positiondetection portion 70.

In a case where the imaging mode is set, image light showing the subjectis formed on a light receiving surface of the color imaging element 20through the imaging lens 18 including the focus lens 16 which is movableby a manual operation, and a mechanical shutter 72.

The device control portion 74 is connected to the CPU 52. In addition,the device control portion 74 is connected to the imaging element 20 andthe mechanical shutter 72. Furthermore, the device control portion 74 isconnected to the motor 50 of the imaging lens 18 through the mounts 44and 46.

The device control portion 74 controls the imaging element 20, themechanical shutter 72, and the motor 50 under control of the CPU 52.

For example, as illustrated in FIG. 4 , the hybrid finder 21 includes anOVF 76 and an EVF 78. The OVF 76 is a reverse Galilean finder includingan objective lens 81 and an eyepiece lens 86, and the EVF 78 includes asecond display 80, a prism 84, and the eyepiece lens 86.

A liquid crystal shutter 88 is arranged in front of the objective lens81. The liquid crystal shutter 88 blocks light such that the opticalimage is not incident on the objective lens 81 in a case of using theEVF 78.

The prism 84 guides the electronic image or various information to bedisplayed on the second display 80 to the eyepiece lens 86 by reflectingthe electronic image or various information, and combines the opticalimage with the electronic image and/or various information to bedisplayed on the second display 80.

In a case where the finder switching lever 23 is rotationally moved inthe direction of the arrow SW illustrated in FIG. 1 , an OVF mode inwhich the optical image is visually recognizable by the OVF 76 and anEVF mode in which the electronic image is visually recognizable by theEVF 78 are alternately switched each time the finder switching lever 23is rotationally moved.

In a case of the OVF mode, the second display control portion 66 enablesthe optical image to be visually recognized from the eyepiece portion bycontrolling the liquid crystal shutter 88 to enter a non-light blockingstate. In a case of the EVF mode, the second display control portion 66enables only the electronic image displayed on the second display 80 tobe visually recognized from the eyepiece portion by controlling theliquid crystal shutter 88 to enter a light blocking state.

The imaging element 20 is one example of a “laminated imaging element”according to the embodiment of the technology of the present disclosure.For example, the imaging element 20 is a complementary metal oxidesemiconductor (CMOS) image sensor. For example, as illustrated in FIG. 5, a photoelectric conversion element 92, a processing circuit 94, and amemory 96 are incorporated in the imaging element 20. In the imagingelement 20, the photoelectric conversion element 92 is laminated withthe processing circuit 94 and the memory 96. The memory 96 is oneexample of a storage portion according to the embodiment of thetechnology of the present disclosure.

The processing circuit 94 is, for example, LSI, and the memory 96 is,for example, a RAM. A dynamic random access memory (DRAM) is employed asone example of the memory 96 in the first embodiment. However, thetechnology of the present disclosure is not limited thereto, and astatic random access memory (SRAM) may be used.

The processing circuit 94 is implemented by an ASIC in the firstembodiment. However, the technology of the present disclosure is notlimited thereto. For example, at least one of a PLD or an FPGA may beemployed instead of the ASIC. Alternatively, at least one of the ASIC,the PLD, or the FPGA may be employed. The CPU may be a single CPU or aplurality of CPUs. Alternatively, a computer including a CPU, a ROM, anda RAM may be employed. Alternatively, the processing circuit 94 may beimplemented by a combination of a hardware configuration and a softwareconfiguration.

The photoelectric conversion element 92 includes a plurality ofphotosensors arranged in a matrix form. In the first embodiment,photodiodes are employed as one example of the photosensors. Photodiodesof “4896×3265” pixels are illustrated as one example of the plurality ofphotosensors.

The photoelectric conversion element 92 comprises color filters, and thecolor filters include a G filter corresponding to green (G) that mostcontributes to obtaining a brightness signal, an R filter correspondingto red (R), and a B filter corresponding to blue (B). In the firstembodiment, the G filter, the R filter, and the B filter are arrangedwith a predetermined periodicity in each of a row direction (horizontaldirection) and a column direction (vertical direction) for the pluralityof photodiodes of the photoelectric conversion element 92. Thus, theimaging apparatus 10 can perform processing in accordance with arepeating pattern in a case of performing demosaicing and the like on R,G, and B signals. The demosaicing refers to processing of calculatingall color information for each pixel from a mosaic image correspondingto color filter arrangement of a single plate color imaging element. Forexample, in a case of an imaging element consisting of color filters ofthree colors of R, G, and B, the demosaicing means processing ofcalculating color information about all of R, G, and B for each pixelfrom a mosaic image consisting of R, G, and B.

While the CMOS image sensor is illustrated as the imaging element 20,the technology of the present disclosure is not limited thereto. Forexample, the technology of the present disclosure is also established ina case where the photoelectric conversion elements 92 is a chargecoupled device (CCD) image sensor.

The imaging element 20 has a so-called electronic shutter function andcontrols an electric charge accumulation time period of each photodiodein the photoelectric conversion element 92 by performing the electronicshutter function under control of the device control portion 74. Theelectric charge accumulation time period refers to a so-called shutterspeed.

The processing circuit 94 is controlled by the device control portion74. The processing circuit 94 reads out captured image data that isobtained by imaging the subject by the photoelectric conversion element92. The “captured image data” here refers to image data indicating thesubject. The captured image data is signal electric charges accumulatedin the photoelectric conversion element 92. The processing circuit 94performs analog/digital (A/D) conversion on the captured image data readout from the photoelectric conversion element 92. The processing circuit94 stores, in the memory 96, the captured image data obtained byperforming the A/D conversion on the captured image data. The processingcircuit 94 acquires the captured image data from the memory 96 andoutputs, to the I/F 56 of the rear stage circuit 90, output image datathat is image data based on the acquired captured image data.Hereinafter, for convenience of description, the “output image data thatis image data based on the captured image data” will be simply referredto as the “output image data”.

The processing circuit 94 performs first processing and secondprocessing on the captured image data. The first processing refers toprocessing of reading out the captured image data from the photoelectricconversion element 92 and storing the read-out captured image data inthe memory 96. The second processing refers to processing of outputtingthe output image data to the outside of the imaging element 20. Forexample, the “outside of the imaging element 20” here refers to the I/F56 of the rear stage circuit 90.

The first processing includes cut-out processing. The cut-out processingrefers to processing of cutting out partial image data indicating animage of a part of the subject in the captured image data from adesignated address in the memory 96 in a state where the captured imagedata is stored in the memory 96. The partial image data is at least oneof a plurality of pieces of divided region image data corresponding to aplurality of divided regions obtained by dividing a part of the subject.

The “plurality of pieces of divided region image data” here have arelationship in which pixels different from each other are thinned outin units of lines. In the first embodiment, horizontal lines are thinnedout in the vertical direction by skipping two lines at a time. However,the technology of the present disclosure is not limited thereto. Forexample, the horizontal lines may be thinned out in the verticaldirection by skipping one line at a time, or the horizontal lines may bethinned out in the vertical direction three or more lines at a time.Vertical lines may be thinned out in the horizontal direction apredetermined number of lines more than or equal to one line at a time.Thinning out may be performed in accordance with a predetermined rulefor each pixel or in units of pixel groups of a few pixels.

As described in detail later, for example, as the “plurality of piecesof divided region image data” here, partial image data 200 (refer toFIGS. 11A and 11B and FIG. 12 ) obtained from the captured image data ofeach of a first frame to a third frame illustrated in FIG. 10 , FIG. 15, or FIG. 16 is illustrated.

The output image data output from the processing circuit 94 is broadlyclassified into the partial image data, wide range image data, andfocusing control image data. The wide range image data refers to imagedata indicating an image of a range of the subject wider than the partof the subject related to the partial image data. For example, the widerange image data refers to entirety image data that indicates an imageof the entirety of the subject in a range falling within an angle ofview imageable by all photodiodes in a partial region 92A (refer toFIGS. 11A and 11B) of the photoelectric conversion element 92. Thefocusing control image data refers to image data indicating an image ofa range that is a range of the subject wider than the part of thesubject related to the partial image data and is a predetermined rangeas a range required for the focusing control of the imaging apparatus10. For example, the “predetermined range” here refers to a range thatis obtained in advance by the CPU 52 by test using an actual apparatusand/or computer simulation or the like as a range which can be used forspecifying a focusing state using so-called contrast AF or phasedifference AF. In the first embodiment, “all photodiodes” refer tousable photodiodes that do not have defects.

In the imaging element 20, the subject is imaged at a first frame rate.The processing circuit 94 performs the first processing at the firstframe rate and performs the second processing at a second frame rate.The first frame rate is a frame rate higher than the second frame rate.In the first embodiment, the second frame rate is 60 frames per second(fps), and the first frame rate is 240 fps. However, the technology ofthe present disclosure is not limited thereto, and a relationship“second frame rate<first frame rate” may be satisfied. In addition, thefirst frame rate is a frame rate that is variable in a range of not lessthan or equal to the second frame rate.

For example, as illustrated in FIG. 6 , the processing circuit 94includes a photoelectric conversion element drive circuit 94A, ananalog-to-digital (AD) conversion circuit 94B, an image processingcircuit 94C, and an output circuit 94D and operates under control of theCPU 52. The photoelectric conversion element drive circuit 94A isconnected to the photoelectric conversion element 92 and the ADconversion circuit 94B. The memory 96 is connected to the AD conversioncircuit 94B and the image processing circuit 94C. The image processingcircuit 94C is connected to the output circuit 94D. The output circuit94D is connected to the I/F 56 of the rear stage circuit 90.

The photoelectric conversion element drive circuit 94A controls thephotoelectric conversion element 92 and reads out analog captured imagedata from the photoelectric conversion element 92 under control of thedevice control portion 74. The AD conversion circuit 94B digitizes thecaptured image data read out by the photoelectric conversion elementdrive circuit 94A and stores the digitized captured image data in thememory 96.

The memory 96 is a memory that can store the captured image data of aplurality of frames. The image processing circuit 94C is one example ofa processing portion according to the embodiment of the technology ofthe present disclosure and performs processing on the captured imagedata. That is, the image processing circuit 94C cuts out the outputimage data from the captured image data stored in the memory 96 byrandom access to the memory 96. The image processing circuit 94Cperforms necessary signal processing on the cut-out output image data.The “random access” here refers to an access method capable of directlyaccessing a location in which target data is stored in the memory 96.

The first processing is performed by the photoelectric conversionelement drive circuit 94A, the AD conversion circuit 94B, the memory 96,and the image processing circuit 94C. That is, processing in thephotoelectric conversion element drive circuit 94A, the AD conversioncircuit 94B, the memory 96, and the image processing circuit 94C isperformed at the first frame rate.

While an example of a form in which processing in the photoelectricconversion element drive circuit 94A, the AD conversion circuit 94B, thememory 96, and the image processing circuit 94C is performed at thefirst frame rate is illustratively described here, the technology of thepresent disclosure is not limited thereto. For example, among readingout by the photoelectric conversion element drive circuit 94A, storageof the captured image data in the memory 96 by the AD conversion circuit94B, and processing by the image processing circuit 94C, at leaststorage of the captured image data in the memory 96 by the AD conversioncircuit 94B may be performed at the first frame rate.

The output circuit 94D performs the second processing. That is, theoutput circuit 94D outputs the output image data subjected to the signalprocessing by the image processing circuit 94C to the I/F 56 of the rearstage circuit 90 at the second frame rate. The output circuit 94D is oneexample of an “output portion” according to the embodiment of thetechnology of the present disclosure.

Next, actions of parts of the imaging apparatus 10 according to theembodiment of the technology of the present disclosure will bedescribed.

Hereinafter, for convenience of description, the first display 40 andthe second display 80 will be referred to as a “display apparatus”without a reference sign unless otherwise necessary to distinguishtherebetween for description. The display apparatus is one example of a“display portion” according to the embodiment of the technology of thepresent disclosure. In addition, hereinafter, for convenience ofdescription, the first display control portion 64 and the second displaycontrol portion 66 will be referred to as a “display control portion”without a reference sign unless otherwise necessary to distinguishtherebetween for description. The display control portion is one exampleof a “control portion” according to the embodiment of the technology ofthe present disclosure. In addition, hereinafter, for convenience ofdescription, a case of displaying the live view image on the displayapparatus will be described.

First, image data generation processing executed by the processingcircuit 94 will be described with reference to FIG. 7 . The image datageneration processing illustrated in FIG. 7 is performed at the firstframe rate by the processing circuit 94. Hereinafter, for convenience ofdescription, the memory 96 will be assumed to be a memory capable ofstoring the captured image data of three frames using a FIFO method.

In the image data generation processing illustrated in FIG. 7 , first,in step S100, for example, the photoelectric conversion element drivecircuit 94A reads out the captured image data from the partial region92A of the photoelectric conversion element 92 illustrated in FIGS. 11Aand 11B, and then, the image data generation processing transitions tostep S102.

For example, the captured image data obtained from the partial region92A of the photoelectric conversion element 92 by executing processingof step S100 is captured image data obtained by imaging the subject bythe partial region 92A of the photoelectric conversion element 92 asillustrated in FIGS. 11A and 11B. While the captured image data obtainedby imaging by the partial region 92A is illustrated here for convenienceof description, the technology of the present disclosure is not limitedthereto. For example, captured image data obtained by imaging thesubject by all photodiodes of the photoelectric conversion element 92may be used, or a captured image obtained by imaging the subject by eachof a plurality of locations in the photoelectric conversion element 92may be used.

For example, the captured image data of one frame read out from thepartial region 92A of the photoelectric conversion element 92 byexecuting processing of step S100 is image data in which the horizontallines are thinned out in the vertical direction by skipping two lines ata time as illustrated in FIG. 10 . In addition, for example, asillustrated in FIG. 10 , in a case where a cycle for reading out thecaptured image data of three frames is one cycle, the captured imagedata read out from the partial region 92A of the photoelectricconversion element 92 is thinned out by shifting the horizontal linesone line at a time in the vertical direction for each frame in onecycle.

While a state example of the captured image data of the first frame tothe third frame is schematically illustrated in the example illustratedin FIG. 10 , pixels from a fourth frame are thinned out using the samemethod as the first frame to the third frame. In addition, while anexample of thinning out in a case where one cycle is three frames isillustrated in the example illustrated in FIG. 10 for convenience ofdescription, the technology of the present disclosure is not limitedthereto, and thinning out may be performed using the same method even ina case where one cycle is four frames or more.

A method of thinning out for the captured image data is not limited to amethod of thinning out the horizontal lines in the vertical direction byshifting one line at a time. For example, a method of thinning out thevertical lines in the horizontal direction by shifting one line at atime may be used, or a method of thinning out in accordance with apredetermined rule in units of a predetermined number of pixels morethan or equal to one pixel may be used.

In step S102, the AD conversion circuit 94B digitizes and stores thecaptured image data read out by the photoelectric conversion elementdrive circuit 94A in the memory 96, and then, the image data generationprocessing transitions to step S104.

In step S104, the image processing circuit 94C determines whether or nota cut-out instruction is received by the touch panel 42 and/or theoperation portion 54. The cut-out instruction refers to an instructionto cut out the partial image data from the captured image data. Thecut-out instruction includes a designation address that is an address inthe memory 96 storing the captured image data and can be used fordesignating a target to be cut out in the captured image data.

In a case where the entirety image in a live view format based on thecaptured image data is displayed on the display apparatus, aninstruction for any location in the entirety image is provided throughthe touch panel 42 and/or the operation portion 54 by the user, andcoordinates of the location of the instruction are converted into anaddress of the memory 96. The address obtained by conversion is thedesignation address. In the example illustrated in FIGS. 11A and 11B, astate where the partial image data 200 is specified from the capturedimage data stored in the memory 96 is illustrated. The partial imagedata 200 is specified by the designation address included in the cut-outinstruction.

A face detection instruction is illustrated as one example of thecut-out instruction. The face detection instruction refers to aninstruction to perform a face detection function. The face detectionfunction is a function of detecting a face of a person who is thesubject from the captured image data, and is implemented by the imageprocessing circuit 94C. For example, as illustrated in FIG. 12 , anaddress of a rectangular region including a face 202F of a person 202who is the subject in the captured image data is specified as an addressof the target to be cut out, and the partial image data 200 is specifiedin accordance with the specified address.

While face detection is illustrated here, a pupil detection function ofdetecting a pupil of the person who is the subject may be performed. Byperforming the face detection function and/or the pupil detectionfunction, a region of a size designated as the target to be cut out froma center of the face of the person who is the subject is decided basedon a face detection result and/or a pupil detection result. Thus, evenin a case where the person who is the subject moves, the CPU 52 canperform AF by following a motion of the person. In addition, byperforming the face detection on the memory 96 and calculating centercoordinates of the face of the person, the display apparatus can performdisplaying in display control processing described later with a smallertime lag in adjustment of an angle of view than in a case of notperforming the face detection and/or the pupil detection. Furthermore,an enlargement ratio of the partial image data 200 may be changed basedon the face detection result and/or the pupil detection result.

In step S104, in a case where the cut-out instruction is received by thetouch panel 42 and/or the operation portion 54, a positive determinationis made, and the image data generation processing transitions to stepS110. In step S104, in a case where the cut-out instruction is notreceived by the touch panel 42 and/or the operation portion 54, anegative determination is made, and the image data generation processingtransitions to step S106.

In step S106, the image processing circuit 94C acquires the capturedimage data from the memory 96, and then, the image data generationprocessing transitions to step S108.

In step S108, the image processing circuit 94C generates the entiretyimage data based on the captured image data acquired in step S106 andoutputs the generated entirety image data to the output circuit 94D, andthen, the image data generation processing transitions to step S130. Theentirety image data generated by executing processing of step S108 isone type of image data included in the output image data.

The cut-out instruction received by the touch panel 42 and/or theoperation portion 54 in step S104 is broadly classified into a low imagequality cut-out instruction and a high image quality cut-outinstruction. The low image quality cut-out instruction refers to aninstruction to cut out a low image quality cut-out image, and the highimage quality cut-out instruction refers to an instruction to cut out ahigh image quality cut-out image. The high image quality cut-out imagehas higher image quality than the low image quality cut-out image. Forexample, the low image quality cut-out image refers to image data inwhich the horizontal lines are thinned out in the vertical direction byskipping two lines at a time as illustrated in FIG. 10 . Meanwhile, forexample, the high image quality cut-out image refers to combined dataobtained by combining the captured image data of three frames, that is,non-thinned image data, as illustrated in FIG. 10 .

In step S110, the image processing circuit 94C determines whether or notthe cut-out instruction received by the touch panel 42 and/or theoperation portion 54 in step S104 is the high image quality cut-outinstruction. In step S110, in a case where the cut-out instructionreceived by the touch panel 42 and/or the operation portion 54 in stepS104 is not the high image quality cut-out instruction, that is, in acase where the cut-out instruction is the low image quality cut-outinstruction, a negative determination is made, and a transition is madeto step S112. In step S110, in a case where the cut-out instructionreceived by the touch panel 42 and/or the operation portion 54 in stepS104 is the high image quality cut-out instruction, a positivedetermination is made, and a transition is made to step S118.

In step S112, the image processing circuit 94C acquires the designationaddress from the cut-out instruction received in step S104, and then,the image data generation processing transitions to step S114.

In step S114, the image processing circuit 94C cuts out the partialimage data from the captured image data of a single frame of thecaptured image data in the memory 96 in accordance with the designationaddress acquired in step S112, and then, the image data generationprocessing transitions to step S116. For example, the captured imagedata of the single frame as a cut-out target for the partial image datais the captured image data least recently stored in the memory 96.However, the technology of the present disclosure is not limitedthereto. For example, in a case where the captured image data of aplurality of frames is stored in the memory 96, the captured image dataof the single frame as the cut-out target for the partial image data maybe any captured image data in the captured image data of the pluralityof frames stored in the memory 96.

While the captured image data of the single frame is used as thecaptured image data of the cut-out target, for example, combined dataobtained by combining the captured image data of two frames in thecaptured image data of the first to third frames illustrated in FIG. 10may be used as the cut-out target. That is, in a case where the combineddata obtained by combining the captured image data of three frames isdefined as high image quality cut-out image data, the captured imagedata of one frame may be used as the captured image data of the cut-outtarget, or the combined data obtained by combining the captured imagedata of two frames may be used as the cut-out target.

In step S116, the image processing circuit 94C generates and outputs lowimage quality cut-out image data to the output circuit 94D, and then,the image data generation processing transitions to step S130. Thepartial image data cut out in step S114 is employed as the low imagequality cut-out image data. However, the technology of the presentdisclosure is not limited thereto. For example, the low image qualitycut-out image data may be image data obtained by performing varioustypes of image processing on the partial image data cut out in stepS114. The low image quality cut-out image data generated by executingprocessing of step S116 is one type of image data included in the outputimage data.

In step S118, the image processing circuit 94C determines whether or notthe captured image data of a plurality of frames is stored in the memory96. In step S118, three frames are employed as one example of theplurality of frames. In step S118, in a case where the captured imagedata of the plurality of frames is not stored in the memory 96, anegative determination is made, and a transition is made to step S120.In step S118, in a case where the captured image data of the pluralityof frames is stored in the memory 96, a positive determination is made,and a transition is made to step S124.

In step S120, the image processing circuit 94C acquires the capturedimage data from the memory 96, and then, the image data generationprocessing transitions to step S122.

In step S122, the image processing circuit 94C generates the entiretyimage data based on the captured image data acquired in step S120 andoutputs the generated entirety image data to the output circuit 94D, andthen, the image data generation processing transitions to step S118. Theentirety image data generated by executing processing of step S122 isone type of image data included in the output image data.

In step S124, the image processing circuit 94C acquires the designationaddress from the cut-out instruction received in step S104, and then,the image data generation processing transitions to step S126.

In step S126, the image processing circuit 94C cuts out the partialimage data from the captured image data of each of the plurality offrames of the captured image data in the memory 96 in accordance withthe designation address acquired in step S124, and then, the image datageneration processing transitions to step S128.

For example, the captured image data of the plurality of frames as thecut-out target for the partial image data is the captured image data ofall frames currently stored in the memory 96, that is, the capturedimage data of three frames. However, the technology of the presentdisclosure is not limited thereto. In a case where the captured imagedata of one frame is defined as the low image quality cut-out imagedata, the combined data obtained by combining the captured image data oftwo frames may be used as the cut-out target.

In step S128, the image processing circuit 94C generates and outputs thehigh image quality cut-out image data to the output circuit 94D, andthen, the image data generation processing transitions to step S130. Forexample, the combined data obtained by combining the partial image dataof three frames cut out in step S126 as illustrated in FIG. 10 isemployed as the high image quality cut-out image data. However, thetechnology of the present disclosure is not limited thereto. Forexample, the high image quality cut-out image data may be image dataobtained by performing various types of image processing on the combineddata obtained by combining the partial image data of three frames cutout in step S126. The high image quality cut-out image data generated byexecuting processing of step S128 is one type of image data included inthe output image data.

In step S130, the image processing circuit 94C determines whether or notan image data generation processing finish condition that is a conditionfor finishing the image data generation processing is satisfied. Forexample, a condition that an instruction to finish the image datageneration processing is received by the touch panel 42 and/or theoperation portion 54 is illustrated as the image data generationprocessing finish condition. In addition, for example, a condition thata time period in which a positive determination is not made in step S104from a start of the image data generation processing exceeds apredetermined time period is illustrated as the image data generationprocessing finish condition. For example, the “predetermined timeperiod” here is five minutes. The predetermined time period may be afixed value or a variable value that can be changed in accordance withan instruction provided from the user.

In step S130, in a case where the image data generation processingfinish condition is not satisfied, a negative determination is made, andthe image data generation processing transitions to step S100. In stepS130, in a case where the image data generation processing finishcondition is satisfied, a positive determination is made, and theprocessing circuit 94 finishes the image data generation processing.

Next, image data output processing executed by the output circuit 94D ofthe processing circuit 94 will be described with reference to FIG. 8 .The image data output processing illustrated in FIG. 8 is performed atthe second frame rate by the output circuit 94D.

In the image data output processing illustrated in FIG. 8 , first, instep S150, the output circuit 94D determines whether or not the outputimage data is input from the image processing circuit 94C. Image datathat includes at least one of the entirety image data, the high imagequality cut-out image data, or the low image quality cut-out image dataoutput by the image data generation processing illustrated in FIG. 7 isillustrated as one example of the output image data input into theoutput circuit 94D from the image processing circuit 94C.

In step S150, in a case where the output image data is not input fromthe image processing circuit 94C, a negative determination is made, anda transition is made to step S154. In step S150, in a case where theoutput image data is input from the image processing circuit 94C, apositive determination is made, and a transition is made to step S152.

In step S152, the output circuit 94D outputs the output image data tothe I/F 56 of the rear stage circuit 90 and then, transitions to stepS154. For example, the I/F 56 outputs the output image data input fromthe output circuit 94D to the CPU 52 and the image processing portion 62through the bus 68.

In step S154, the output circuit 94D determines whether or not an imagedata output processing finish condition that is a condition forfinishing the image data output processing is satisfied. For example,the image data output processing finish condition is the same conditionas the image data generation processing finish condition.

In step S154, in a case where the image data output processing finishcondition is not satisfied, a negative determination is made, and theimage data output processing transitions to step S150. In step S154, ina case where the image data output processing finish condition issatisfied, a positive determination is made, and the output circuit 94Dfinishes the image data output processing.

Next, the display control processing executed by the display controlportion of the rear stage circuit 90 will be described with reference toFIG. 9 . For convenience of description, it is assumed that the outputimage data is output to the rear stage circuit 90 from the outputcircuit 94D by executing the image data output processing illustrated inFIG. 8 , and that the output image data is input into the CPU 52 and theimage processing portion 62. In addition, hereinafter, for convenienceof description, the high image quality cut-out image data and the lowimage quality cut-out image data will be referred to as “cut-out imagedata” unless otherwise necessary to distinguish therebetween fordescription. In addition, the high image quality cut-out image indicatedby the high image quality cut-out image data and the low image qualitycut-out image indicated by the low image quality cut-out image data willbe referred to as a “cut-out image” unless otherwise necessary todistinguish therebetween for description.

In the display control processing illustrated in FIG. 9 , in step S160,the display control portion determines whether or not the output imagedata is input from the image processing portion 62. In step S160, in acase where the output image data is not input from the image processingportion 62, a negative determination is made, and the display controlprocessing transitions to step S164. In step S160, in a case where theoutput image data is input from the image processing portion 62, apositive determination is made, and the display control processingtransitions to step S162.

In step S162, the display control portion outputs the output image datato the display apparatus as graphics data, and then, the display controlprocessing transitions to step S164. In a case where the output imagedata is output to the display apparatus by executing processing of stepS162, the display apparatus displays an image indicated by the outputimage data. For example, in a case where the entirety image data isincluded in the output image data, the display apparatus displays theentirety image indicated by the entirety image data. The “entiretyimage” here refers to an image showing the entirety of the subject inthe range falling within the angle of view imageable by all photodiodesin the partial region 92A of the photoelectric conversion element 92.

For example, in a case where the cut-out image data is included in theoutput image data, the display apparatus displays the cut-out imageindicated by the cut-out image data as illustrated in FIGS. 11A and 11Band FIG. 12 .

For example, in a case where an enlarged display instruction is receivedby the touch panel 42 and/or the operation portion 54, the displayapparatus displays the cut-out image in an enlarged manner asillustrated in FIG. 14 .

The cut-out image is any of the high image quality cut-out image and thelow image quality cut-out image. For example, the high image qualitycut-out image is an image based on the combined data obtained bycombining the captured image data of three frames as illustrated in FIG.10 . Meanwhile, the low image quality cut-out image is a thinned imagebased on the captured image data of one frame. The entirety image isalso a thinned image based on the captured image data of one frame.Thus, image quality of the image displayed on the display apparatus issuch that the high image quality cut-out image has higher image qualitythan the low image quality cut-out image and the entirety image.

It is not necessary to display the same image on both of the firstdisplay 40 and the second display 80. For example, the entirety imagemay be displayed on one of the first display 40 and the second display80, and the cut-out image may be displayed on the other together withthe entirety image. Alternatively, the cut-out image may be displayedwithout displaying the entirety image. In a case of displaying thecut-out image on the display apparatus together with the entirety image,first to third display methods are considered. The first display methodrefers to a display method of arranging and displaying the entiretyimage and the cut-out image. The second display method refers to adisplay method of displaying by superimposing one of the entirety imageand the cut-out image on the other. The third display method refers to adisplay method of displaying by embedding one of the entirety image andthe cut-out image in a display region of the other.

In step S164, the display control portion determines whether or not adisplay control processing finish condition that is a condition forfinishing the display control processing is satisfied. For example, thedisplay control processing finish condition is the same condition as theimage data generation processing finish condition.

In step S164, in a case where the display control processing finishcondition is not satisfied, a negative determination is made, and thedisplay control processing transitions to step S160. In step S164, in acase where the display control processing finish condition is satisfied,a positive determination is made, and the display control portionfinishes the display control processing.

For example, by executing the image data generation processingillustrated in FIG. 7 , the image data output processing illustrated inFIG. 8 , and the display control processing illustrated in FIG. 9 , thelive view image is displayed on the display apparatus under control ofthe display control portion. The “live view image” here refers to animage in a live view format, that is, a live preview image, based on theoutput image data including at least one of the cut-out image data orthe entirety image data. In the imaging apparatus 10 according to thefirst embodiment, the live view image is displayed at the second framerate.

The imaging apparatus 10 according to the first embodiment is differentfrom an imaging apparatus according to a technology in the related artin that a laminated CMOS image sensor is employed as the imaging element20. That is, in an imaging element according to the technology in therelated art, the laminated CMOS image sensor is not employed. Thus, inthe imaging apparatus according to the technology in the related art,for example, as illustrated in FIG. 11A, the captured image dataobtained by imaging the subject by the imaging element is output to therear stage circuit 90, and the partial image data 200 is cut out fromthe captured image data by the rear stage circuit 90. An image based onthe cut-out partial image data 200 is displayed on the displayapparatus.

Meanwhile, in the imaging apparatus 10 according to the firstembodiment, the laminated CMOS image sensor is employed as the imagingelement 20. Thus, for example, as illustrated in FIG. 11B, the capturedimage data is temporarily stored in the memory 96 incorporated in theimaging element 20, and the partial image data 200 is cut out by randomaccess to the memory 96. Image data based on the cut-out partial imagedata 200 is output to the rear stage circuit 90. That is, a data amountoutput to the rear stage circuit 90 from the imaging element 20according to the first embodiment is smaller than a data amount outputto the rear stage circuit 90 from the imaging element of the imagingapparatus according to the technology in the related art. Thus, theimaging apparatus 10 according to the first embodiment can reduce powerconsumption accompanied by output of the image data compared to theimaging apparatus according to the technology in the related art.

Since the partial image data 200 is cut out in the rear stage circuit90, for example, it is necessary to move the captured image data of thefirst frame to the rear stage circuit 90 after first exposure forobtaining the captured image data of the first frame is finished asillustrated in FIG. 13A. The captured image data is moved to the rearstage circuit 90 from the imaging element at the second frame rate.

Meanwhile, for example, as illustrated in FIG. 13B, in the imagingapparatus 10, image processing such as read-out of the captured imagedata, storage of the captured image data in the memory 96, and cut-outof the partial image data 200 is performed at the first frame rate inthe imaging element 20. Even in the imaging apparatus 10, the capturedimage data is moved to the rear stage circuit 90 from the imagingelement 20 at the second frame rate.

However, in the imaging apparatus 10, at least imaging processing fromread-out of the captured image data to cut-out or the like is performedat the first frame rate. Thus, for example, as illustrated in FIG. 13B,the imaging apparatus 10 can perform more processing in the imagingelement 20 than the imaging apparatus according to the technology in therelated art even during movement of the output image data to the rearstage circuit 90. In the example illustrated in FIG. 13B, secondexposure and image processing from read-out of the captured image dataof the second frame obtained by the second exposure to cut-out or thelike are performed during movement of the output image data of the firstframe to the rear stage circuit 90 from the imaging element 20.

In addition, in the imaging apparatus 10, for example, as illustrated inFIG. 10 , the captured image data of the plurality of frames is storedin the memory 96 in the imaging element 20. Thus, the processing circuit94 can output combined data obtained by combining the captured imagedata of the plurality of frames to the rear stage circuit 90.

In the example illustrated in FIG. 10 , the captured image data of eachof three consecutive frames is image data obtained by thinning out thehorizontal lines in the vertical direction by shifting one line at atime. However, the technology of the present disclosure is not limitedthereto. For example, as illustrated in FIG. 15 , image data obtained bynot thinning out the captured image data of each of three consecutiveframes may be used. In this case, for example, the amount of electriccharges per line of the horizontal lines used for exposure in thephotoelectric conversion element 92 is ⅓ of the amount of electriccharges per line of the horizontal lines used for exposure for obtainingthe captured image data of each frame illustrated in FIG. 10 .Accordingly, even in a case of the example illustrated in FIG. 15 , theoutput image data that can implement the same image quality as in a caseof the example illustrated in FIG. 10 is obtained by combining thecaptured image data of the first frame to the third frame.

Here, “⅓ of the amount of electric charges per line of the horizontallines used for exposure for obtaining the captured image data of eachframe” corresponds to a ratio of the amount of electric charges obtainedby exposure using a second exposure method described later, to theamount of electric charges obtained by exposure using a first exposuremethod described later.

In the imaging apparatus 10 according to the first embodiment, forexample, as illustrated in FIG. 18A and FIG. 18B, the first exposuremethod and the second exposure method are selectively used in accordancewith an instruction received by the touch panel 42 and/or the operationportion 54. Hereinafter, for convenience of description, a time periodrequired for exposure performed using the first exposure method will bereferred to as a first exposure time period, and a time period requiredfor exposure performed using the second exposure method will be referredto as a second exposure time period.

For example, in the first exposure method illustrated in FIG. 18A, thesubject is imaged in the first exposure time period by all photodiodesin the partial region 92A of the photoelectric conversion element 92illustrated in FIG. 11B. Accordingly, for example, image data havingbrightness corresponding to brightness of the output image data obtainedby combining the first frame to the third frame illustrated in FIG. 15is obtained.

Meanwhile, for example, in the second exposure method illustrated inFIG. 18B, while the subject is imaged in the second exposure time periodby all photodiodes in the partial region 92A of the photoelectricconversion element 92, the second exposure time period is ⅓ of the firstexposure time period. Thus, for example, as illustrated in FIG. 15 , theamount of electric charges of the captured image data of one frame is ⅓of the amount of electric charges of the captured image data of oneframe based on the first exposure method.

In the first exposure method, the number of times that a series ofprocessing of image processing such as read-out of the captured imagedata from the photoelectric conversion element 92, storage of thecaptured image data in the memory 96, and cut-out of the partial imagedata is performed is three. That is, the number of times that the firstprocessing is performed is larger in the second exposure method than inthe first exposure method by two. The number of times that the firstprocessing is performed in the second exposure method is a number oftimes that can be implemented at the first frame rate. Thus, outputimage data having brightness corresponding to brightness of the outputimage data of one frame obtained in the first exposure time period canbe generated without changing the second exposure time period to thefirst exposure time period by extending the exposure time period bydecreasing the first frame rate. That is, even in the second exposuremethod, the output image data having brightness corresponding to thebrightness of the output image data of one frame obtained using thefirst exposure method can be generated by obtaining the cut-out imagefrom the captured image data of each of three consecutive frames andcombining the cut-out images.

In addition, for example, captured image data in which the horizontallines are thinned out in the vertical direction by skipping two lines ata time in the captured image data of each of the first frame to thethird frame illustrated in FIG. 16 may be employed. In the exampleillustrated in FIG. 16 , the captured image data in which the horizontallines are thinned out in the vertical direction by skipping two lines ata time in the captured image data of each of the first frame to thethird frame is illustrated. In the example illustrated in FIG. 16 ,output image data in which the horizontal lines are thinned out in thevertical direction by skipping two lines at a time is obtained bycombining the captured image data of the first frame to the third frame.

In addition, in the imaging apparatus 10 according to the firstembodiment, since the laminated CMOS sensor is employed as the imagingelement 20, the image quality can be changed even in a case where thetotal data amount is the same, by changing a method of reading out theoutput image data from the captured image data of the memory 96.

For example, as illustrated in FIG. 17 , a data amount of the outputimage data acquired from the memory 96 by the image processing circuit94C can be the same between a case of thinning out and a case of notthinning out. For example, as illustrated in FIG. 17 , non-thinnedcaptured image data does not need to be thinned out as much as aread-out target is narrower than thinned captured image data. Thus, thenon-thinned captured image data can implement higher image quality thanthe thinned captured image. In the imaging apparatus 10, for example, asillustrated in FIG. 17 , regardless of whether or not thinning isperformed, a data amount output from the processing circuit 94 issmaller than in a case of outputting the captured image data of allpixels to the rear stage circuit 90. Thus, in the imaging apparatus 10,power consumption required for output to the rear stage circuit 90 fromthe output circuit 94D can be reduced compared to a case of performingthinning processing or cut-out processing in the rear stage circuit 90as in the imaging apparatus according to the technology in the relatedart.

As described above, in the imaging apparatus 10, the memory 96 and theprocessing circuit 94 are incorporated in the imaging element 20. Inaddition, the processing circuit 94 executes the first processing at thefirst frame rate and executes the second processing at the second framerate. The first processing includes the cut-out processing. In addition,for example, the partial image data is cut out from the captured imagedata by performing the cut-out processing as in steps S114 and S116illustrated in FIG. 7 by the processing circuit 94. The output imagedata includes image data based on the cut-out partial image data.

Accordingly, the imaging apparatus 10 can reduce power consumptioncompared to a case of cutting out the partial image data from image dataoutput to the outside of the imaging element without passing through thestorage portion, and outputting the partial image data.

In addition, in the imaging apparatus 10, the image processing circuit94C cuts out the partial image data from the captured image data storedin the memory 96 by random access to the memory 96.

Accordingly, the imaging apparatus 10 can quickly cut out the partialimage data from the designated address compared to a case where randomaccess to the memory 96 is not available.

In addition, in the imaging apparatus 10, the memory 96 can store thecaptured image data in a plurality of frames, and the cut-out processingincludes processing of cutting out the partial image data of theplurality of frames from the captured image data stored in the memory96. In addition, for example, as in processing of steps S126 and S128illustrated in FIG. 7 , the second processing includes processing ofcombining the partial image data of the plurality of frames cut out byperforming the cut-out processing by the image processing circuit 94C.Combined data obtained by combining the partial image data of theplurality of frames is output to the rear stage circuit 90 by the outputcircuit 94D as the output image data.

Accordingly, the imaging apparatus 10 can reduce power consumptioncompared to a case of outputting the captured image data to the rearstage circuit 90, cutting out the partial image data of the plurality offrames from the captured image data in the rear stage circuit 90, andcombining the partial image data of the plurality of frames.

In addition, in the imaging apparatus 10, for example, the partial imagedata is cut out from the captured image data of the single frame, andthe cut-out partial image data is employed as the low image qualitycut-out image data illustrated in step S116 in FIG. 7 .

Accordingly, in the imaging apparatus 10, power consumption can bereduced compared to a case of cutting out the partial image data fromthe captured image data of each of the plurality of frames and combiningthe cut-out partial image data of the plurality of frames.

In addition, in the imaging apparatus 10, the partial image data of theplurality of frames has a relationship in which pixels different fromeach other are thinned out in units of lines.

Accordingly, the imaging apparatus 10 can obtain a subject image of highreproducibility compared to a case of thinning out pixels at the sameposition in units of lines.

A degree of thinning may be decided in accordance with a degree ofenlargement of the cut-out image. For example, as the degree ofenlargement is increased, the degree of thinning may be decreased. In acase of performing a special type of processing on the output imagedata, the degree of thinning may be decided in accordance with the typeof processing.

In addition, in the imaging apparatus 10, the output image data outputfrom the output circuit 94D is the combined data obtained by combiningthe partial image data of the plurality of frames thinned out atdifferent locations.

Accordingly, the imaging apparatus 10 can display an image of high imagequality compared to a case of outputting the partial image data of thesingle frame.

In addition, in the imaging apparatus 10, for example, the displaycontrol portion performs a control for displaying, on the displayapparatus, an image based on the output image data output by the outputcircuit 94D in an enlarged manner as illustrated in FIGS. 11A and 11B,FIG. 12 , and FIG. 14 .

Accordingly, the imaging apparatus 10 can cause the user to visuallyrecognize the image based on the output image data output by the outputcircuit 94D.

In addition, in the imaging apparatus 10, the laminated CMOS imagesensor is employed as the imaging element 20. Accordingly, the imagingapparatus 10 can output image data subjected to image processing in theimaging element 20 to the rear stage circuit 90.

While an example of a form in which the captured image data obtained byimaging the subject by the partial region 92A of the photoelectricconversion element 92 is stored in the memory 96 is illustrativelydescribed in the first embodiment, the technology of the presentdisclosure is not limited thereto. For example, as illustrated in FIG.19 , the entirety image data that is the captured image data obtained byimaging the subject by all photodiodes in the photoelectric conversionelement 92 may be stored in the memory 96. Even in this case, the imageprocessing circuit 94C generates the cut-out image data by cutting outthe partial image data 200 from the entirety image data by random accessto the memory 96 in the same manner as the first embodiment. An imagebased on the cut-out image data is displayed on the display apparatus.

For example, as illustrated in FIG. 20 , the entirety image data that isone example of the “wide range image data” according to the embodimentof the technology of the present disclosure, and the cut-out image databased on the partial image data 200 may be output to the rear stagecircuit 90 from the output circuit 94D of the imaging element 20. Inthis case, resolution of the entirety image data is preferably lowerthan resolution of the partial image data. Accordingly, powerconsumption is reduced.

For example, as illustrated in FIG. 20 , the display apparatus maydisplay the cut-out image indicated by the cut-out image data in anenlarged manner and display the entirety image indicated by the entiretyimage data using a display method based on the first display methodunder control of the display control portion.

The display apparatus may display the cut-out image and the entiretyimage using a display method based on the second or third display methodunder control of the display control portion. The display apparatus maydisplay the cut-out image or the entirety image under control of thedisplay control portion. For example, any of the cut-out image and theentirety image that is to be displayed on the display apparatus may bedecided in accordance with an instruction received by the touch panel 42and/or the operation portion 54.

The processing circuit 94 may selectively output a partial image of apart of the partial image data of three frames and the combined dataobtained by combining the partial image data of three frames to the rearstage circuit 90 in accordance with a provided condition as the partialimage data. Accordingly, power consumption is reduced compared to a caseof outputting the combined data to the rear stage circuit 90 at alltimes. For example, the “partial image of the part of the partial imagedata of three frames” refers to a number of partial images less thanthree frames. In addition, for example, the “provided condition” isillustrated by a condition that a specific instruction is received bythe touch panel 42 and/or the operation portion 54, or a condition thatremaining power of the imaging apparatus 10 is below a threshold value.For example, the “condition that the specific instruction is received”here is illustrated by a condition that an instruction for a powersaving mode is received, or a condition that an instruction for imageprocessing that requires a special type of processing is received by thetouch panel 42 and/or the operation portion 54.

Second Embodiment

While a case of cutting out the partial image data indicating the imageof the part of the subject is described in the first embodiment, a casewhere a part of the subject is present across a plurality of regionswill be described in a second embodiment. In the second embodiment, thesame constituents as the first embodiment will be designated by the samereference signs and will not be described.

For example, the imaging apparatus 10 according to the second embodimentis different from the imaging apparatus 10 according to the firstembodiment in that a partial region 92B is included instead of thepartial region 92A of the photoelectric conversion element 92 asillustrated in FIG. 24 . The partial region 92B is a region that has alarger width than the partial region 92A in the vertical direction ofthe photoelectric conversion element 92.

Next, the image data generation processing executed by the processingcircuit 94 according to the second embodiment will be described withreference to FIG. 21 .

In the image data generation processing illustrated in FIG. 21 , first,in step S250, the photoelectric conversion element drive circuit 94Areads out the captured image data from the partial region 92B of thephotoelectric conversion element 92, and then, the image data generationprocessing transitions to step S252.

For example, the captured image data of one frame read out from thepartial region 92B of the photoelectric conversion element 92 byexecuting processing of step S250 is image data in which the horizontallines are thinned out in the vertical direction by skipping two lines ata time as illustrated in FIG. 10 .

For example, the captured image data obtained from the partial region92B of the photoelectric conversion element 92 by executing processingof step S250 is captured image data obtained by imaging the subject bythe partial region 92B of the photoelectric conversion element 92 in thesame manner as the example illustrated in FIGS. 11A and 11B.

In step S252, the AD conversion circuit 94B digitizes and stores thecaptured image data read out by the partial region 92B of thephotoelectric conversion element drive circuit 94A in the memory 96, andthen, the image data generation processing transitions to step S254.

In step S254, the image processing circuit 94C determines whether or notthe cut-out instruction is received by the touch panel 42 and/or theoperation portion 54. In step S254, the “cut-out instruction” is broadlyclassified into a single region cut-out instruction and a plural regioncut-out instruction. The single region cut-out instruction refers to aninstruction to cause the image processing circuit 94C to cut out asingle region from an image indicated by the captured image data. Theplural region cut-out instruction refers to an instruction to cause theimage processing circuit 94C to cut out a plurality of regions from theimage indicated by the captured image data.

The single region cut-out instruction is broadly classified into asingle low image quality image cut-out instruction and a single highimage quality image cut-out instruction. The single low image qualityimage cut-out instruction refers to an instruction to cut out a singlelow image quality image from the image indicated by the captured imagedata as an image of a cut-out region. The single high image qualityimage cut-out instruction refers to an instruction to cut out a singlehigh image quality image from the image indicated by the captured imagedata as an image of a cut-out region. The single high image qualityimage has higher image quality than the single row image quality image.The single low image quality image refers to image data in which thehorizontal lines are thinned out in the vertical direction by skippingtwo lines at a time in the same manner as the example illustrated inFIG. 10 . Meanwhile, the single high image quality image refers to thecombined data obtained by combining the captured image data of threeframes, that is, non-thinned image data, in the same manner as theexample illustrated in FIG. 10 .

The plural region cut-out instruction is broadly classified into aplural low image quality cut-out instruction and a plural high imagequality image cut-out instruction. The plural low image quality cut-outinstruction refers to an instruction to cut out a low image qualityimage from each of the plurality of regions cut out from the imageindicated by the captured image data. Hereinafter, for convenience ofdescription, the low image quality image cut out from each of theplurality of regions will be referred to as the “low image qualitycut-out image”.

The plural high image quality image cut-out instruction refers to aninstruction to cut out a high image quality image from each of theplurality of regions cut out from the image indicated by the capturedimage data. Hereinafter, for convenience of description, the high imagequality image cut out from each of the plurality of regions will bereferred to as the “high image quality cut-out image”.

The high image quality cut-out image has higher image quality than thelow image quality cut-out image. The low image quality cut-out imagerefers to a plurality of pieces of image data in each of which thehorizontal lines are thinned out in the vertical direction by skippingtwo lines at a time in the same manner as the example illustrated inFIG. 10 . Meanwhile, a plurality of high image quality cut-out imagesrefer to a plurality of pieces of combined data each of which isobtained by combining the captured image data of three frames, that is,a plurality of pieces of non-thinned image data, in the same manner asthe example illustrated in FIG. 10 .

In step S254, in a case where the cut-out instruction is received by thetouch panel 42 and/or the operation portion 54, a positive determinationis made, and the image data generation processing transitions to stepS260. In step S254, in a case where the cut-out instruction is notreceived by the touch panel 42 and/or the operation portion 54, anegative determination is made, and the image data generation processingtransitions to step S256.

In step S256, the image processing circuit 94C acquires the capturedimage data from the memory 96, and then, the image data generationprocessing transitions to step S258.

In step S258, the image processing circuit 94C generates main subjectimage data based on the captured image data acquired in step S256 andoutputs the generated main subject image data to the output circuit 94D,and then, the image data generation processing transitions to step S266.For example, the main subject image data generated by executingprocessing of step S258 refers to the subject included in a rectangularregion including faces of a plurality of persons. The faces of thepersons are specified by performing the face detection function providedin the imaging apparatus 10. The main subject image data is one type ofimage data that is a part of the captured image data and is included inthe output image data.

In step S260, the image processing circuit 94C determines whether or notthe cut-out instruction received in step S254 is the plural regioncut-out instruction. In step S260, in a case where the cut-outinstruction received in step S254 is not the plural region cut-outinstruction, that is, in a case where the cut-out instruction receivedin step S254 is the single region cut-out instruction, a negativedetermination is made, and the image data generation processingtransitions to step S262. In step S260, in a case where the cut-outinstruction received in step S254 is the plural region cut-outinstruction, a positive determination is made, and the image datageneration processing transitions to step S264.

In step S262, for example, the image processing circuit 94C executessingle region cut-out processing illustrated in FIG. 22 , and then, theimage data generation processing transitions to step S266.

In step S264, for example, the image processing circuit 94C executesplural region cut-out processing illustrated in FIG. 23 , and then, theimage data generation processing transitions to step S266.

In the single region cut-out processing illustrated in FIG. 22 , in stepS262A, a determination as to whether or not the single region cut-outinstruction received in step S254 is the single high image quality imagecut-out instruction is performed. In step S262A, in a case where thesingle region cut-out instruction received in step S254 is the singlelow image quality image cut-out instruction, a negative determination ismade, and the single region cut-out processing transitions to stepS262B. In step S262A, in a case where the single region cut-outinstruction received in step S254 is the single high image quality imagecut-out instruction, a positive determination is made, and the singleregion cut-out processing transitions to step S262E.

In step S262B, the image processing circuit 94C acquires the designationaddress from the cut-out instruction received in step S254, and then,the single region cut-out processing transitions to step S262C.

In step S262C, the image processing circuit 94C cuts out region imagedata from the captured image data of one frame of the captured imagedata in the memory 96 in accordance with the designation addressacquired in step S262B, and then, the single region cut-out processingtransitions to step S262D. The region image data refers to image data ofthe captured image data that indicates a region specified in accordancewith the designation address in the image indicated by the capturedimage data. For example, the captured image data of the single frame asa cut-out target for the region image data is the captured image dataleast recently stored in the memory 96. However, the technology of thepresent disclosure is not limited thereto. For example, the capturedimage data of the single frame as the cut-out target for the regionimage data may be any captured image data in a case where the capturedimage data of the plurality of frames is stored in the memory 96.

In step S262D, the image processing circuit 94C generates and outputssingle low image quality cut-out image data to the output circuit 94D,and then, the image processing circuit 94C finishes the single regioncut-out processing. The region image data cut out in step S262C isemployed as the single low image quality cut-out image data. However,the technology of the present disclosure is not limited thereto. Forexample, the single low image quality cut-out image data may be imagedata obtained by performing various types of image processing on theregion image data cut out in step S262C. The single low image qualitycut-out image data generated by executing processing of step S262D isone type of image data included in the output image data and is oneexample of “image data based on partial image data” according to theembodiment of the technology of the present disclosure.

In step S262E, the image processing circuit 94C determines whether ornot the captured image data of a plurality of frames is stored in thememory 96. In step S262E, three frames are employed as one example ofthe plurality of frames. In step S262E, in a case where the capturedimage data of the plurality of frames is not stored in the memory 96, anegative determination is made, and a transition is made to step S262F.In step S262E, in a case where the captured image data of the pluralityof frames is stored in the memory 96, a positive determination is made,and a transition is made to step S262H.

In steps S262F and S262G, the image processing circuit 94C executesprocessing corresponding to processing of steps S256 and S258illustrated in FIG. 21 , and the single region cut-out processingtransitions to step S262E.

In step S262H, the image processing circuit 94C acquires the designationaddress from the cut-out instruction received in step S254, and then,the single region cut-out processing transitions to step S262I.

In step S262I, the image processing circuit 94C cuts out the regionimage data from the captured image data of each of the plurality offrames of the captured image data in the memory 96 in accordance withthe designation address acquired in step S262H, and then, the singleregion cut-out processing transitions to step S262J.

For example, the captured image data of the plurality of frames as thecut-out target for the region image data is the captured image data ofall frames currently stored in the memory 96, that is, the capturedimage data of three frames. However, the technology of the presentdisclosure is not limited thereto. In a case where the captured imagedata of one frame is defined as the single low image quality cut-outimage data, the combined data obtained by combining the captured imagedata of two frames may be used as the cut-out target.

The region image data cut out in each of steps S262C and S262I is oneexample of “partial image data” according to the embodiment of thetechnology of the present disclosure.

In step S262J, the image processing circuit 94C generates and outputssingle high image quality cut-out image data to the output circuit 94D,and then, the image processing circuit 94C finishes the single regioncut-out processing. For example, the combined data obtained by combiningthe region image data of three frames cut out in step S262I is employedas the single high image quality cut-out image data in the same manneras the example illustrated in FIG. 10 . However, the technology of thepresent disclosure is not limited thereto. For example, the single highimage quality cut-out image data may be image data obtained byperforming various types of image processing on the combined dataobtained by combining the region image data of three frames cut out instep S262I. The single high image quality cut-out image data generatedby executing processing of step S262J is one type of image data includedin the output image data and is one example of the “image data based onthe partial image data” according to the embodiment of the technology ofthe present disclosure.

In the plural region cut-out processing illustrated in FIG. 23 , in stepS264A, the image processing circuit 94C determines whether or not theplural region cut-out instruction received in step S254 is the pluralhigh image quality image cut-out instruction. In step S264A, in a casewhere the plural region cut-out instruction received in step S254 is theplural low image quality image cut-out instruction, a negativedetermination is made, and the plural region cut-out processingtransitions to step S264B. In step S264A, in a case where the pluralregion cut-out instruction received in step S254 is the plural highimage quality image cut-out instruction, a positive determination ismade, and the plural region cut-out processing transitions to stepS264F.

In step S264B, the image processing circuit 94C acquires a non-processeddesignation address from the cut-out instruction received in step S254,and then, the plural region cut-out processing transitions to stepS264C. In step S264B, the “non-processed designation address” refers toa designation address that is not used yet in processing of step S264Cdescribed later.

In step S264C, the image processing circuit 94C cuts out the regionimage data from the captured image data of one frame of the capturedimage data in the memory 96 in accordance with the designation addressacquired in step S264B, and then, the plural region cut-out processingtransitions to step S264D. For example, the captured image data of thesingle frame as a cut-out target for the region image data is thecaptured image data least recently stored in the memory 96. However, thetechnology of the present disclosure is not limited thereto. Forexample, the captured image data of the single frame as the cut-outtarget for the region image data may be any captured image data in acase where the captured image data of the plurality of frames is storedin the memory 96.

In step S264D, the image processing circuit 94C determines whether ornot all designation addresses are processed. In step S264D, in a casewhere the non-processed designation address is present, a negativedetermination is made, and the plural region cut-out processingtransitions to step S264B. In step S264D, in a case where thenon-processed designation address is not present, a positivedetermination is made, and the plural region cut-out processingtransitions to step S264E.

A plurality of pieces of region image data cut out by executingprocessing of step S264B to step S264D are one example of a “pluralityof pieces of region image data indicating a plurality of regions incaptured image data” according to the embodiment of the technology ofthe present disclosure.

In step S264E, the image processing circuit 94C generates and outputsplural low image quality cut-out image data to the output circuit 94D,and then, the image processing circuit 94C finishes the plural regioncut-out processing. The plurality of pieces of region image data cut outin step S264C are employed as the plural low image quality cut-out imagedata. However, the technology of the present disclosure is not limitedthereto. For example, the plural low image quality cut-out image datamay be image data obtained by performing various types of imageprocessing on the plurality of pieces of region image data cut out instep S264C. The plural low image quality cut-out image data generated byexecuting processing of step S264E is one type of image data included inthe output image data and is one example of the “image data based on thepartial image data” according to the embodiment of the technology of thepresent disclosure.

In step S264F, the image processing circuit 94C determines whether ornot the captured image data of a plurality of frames is stored in thememory 96. In step S264F, three frames are employed as one example ofthe plurality of frames. In step S264F, in a case where the capturedimage data of the plurality of frames is not stored in the memory 96, anegative determination is made, and a transition is made to step S264G.In step S264F, in a case where the captured image data of the pluralityof frames is stored in the memory 96, a positive determination is made,and a transition is made to step 264I.

In steps S264G and S264H, the image processing circuit 94C executesprocessing corresponding to processing of steps S256 and S258illustrated in FIG. 21 , and the plural region cut-out processingtransitions to step S264F.

In step S264I, the image processing circuit 94C acquires thenon-processed designation address from the cut-out instruction receivedin step S254, and then, the plural region cut-out processing transitionsto step S264J. In step S264I, the “non-processed designation address”refers to a designation address that is not used yet in processing ofstep S264J described later.

In step S264J, the image processing circuit 94C cuts out the regionimage data from the captured image data of each of the plurality offrames of the captured image data in the memory 96 in accordance withthe designation address acquired in step S264I, and then, the pluralregion cut-out processing transitions to step S264K.

For example, the captured image data of the plurality of frames as thecut-out target for the region image data is the captured image data ofall frames currently stored in the memory 96, that is, the capturedimage data of three frames. However, the technology of the presentdisclosure is not limited thereto. In a case where the captured imagedata of one frame is defined as the plural low image quality cut-outimage data, the combined data obtained by combining the captured imagedata of two frames may be used as the cut-out target.

In step S264K, the image processing circuit 94C determines whether ornot all designation addresses are processed. In step S264K, in a casewhere the non-processed designation address is present, a negativedetermination is made, and the plural region cut-out processingtransitions to step S264I. In step S264K, in a case where thenon-processed designation address is not present, a positivedetermination is made, and the plural region cut-out processingtransitions to step S264L.

A plurality of pieces of region image data cut out by executingprocessing of step S264I to step S264K are one example of the “pluralityof pieces of region image data indicating the plurality of regions inthe captured image data” according to the embodiment of the technologyof the present disclosure.

A plurality of pieces of region image data cut out by executingprocessing of step S264J are one example of the “plurality of pieces ofregion image data indicating the plurality of regions in the capturedimage data” according to the embodiment of the technology of the presentdisclosure.

In step S264L, the image processing circuit 94C generates and outputsplural high image quality cut-out image data to the output circuit 94D,and then, the image processing circuit 94C finishes the plural regioncut-out processing. For example, the combined data obtained by combiningthe region image data of three frames cut out in step S264J for each ofa plurality of locations as the cut-out target is employed as the pluralhigh image quality cut-out image data in the same manner as the exampleillustrated in FIG. 10 . However, the technology of the presentdisclosure is not limited thereto.

For example, the plural high image quality cut-out image data may beimage data obtained by performing various types of image processing onthe combined data obtained by combining the region image data of threeframes cut out in step S264J for each of the plurality of regions as thecut-out target. The plural high image quality cut-out image datagenerated by executing processing of step S264L is one type of imagedata included in the output image data and is one example of the “imagedata based on the partial image data” according to the embodiment of thetechnology of the present disclosure.

In the image data generation processing illustrated in FIG. 21 , in stepS266, a determination as to whether or not the image data generationprocessing finish condition described in the first embodiment issatisfied is performed. In step S266, in a case where the image datageneration processing finish condition is not satisfied, a negativedetermination is made, and the image data generation processingtransitions to step S250. In step S266, in a case where the image datageneration processing finish condition is satisfied, a positivedetermination is made, and the image processing circuit 94C finishes theimage data generation processing.

Next, the image data output processing executed by the output circuit94D of the processing circuit 94 according to the second embodiment willbe described with reference to FIG. 8 . The image data output processingillustrated in FIG. 8 is performed at the second frame rate by theoutput circuit 94D.

In the image data output processing illustrated in FIG. 8 , first, instep S280, the output circuit 94D determines whether or not the outputimage data is input from the image processing circuit 94C. Image datathat includes at least one of the main subject image data, the singlelow image quality cut-out image data, the single high image qualitycut-out image data, the plural low image quality cut-out image data, orthe plural high image quality cut-out image data output by the imagedata generation processing illustrated in FIG. 21 is illustrated as oneexample of the output image data input into the output circuit 94D fromthe image processing circuit 94C.

In step S280, in a case where the output image data is not input fromthe image processing circuit 94C, a negative determination is made, anda transition is made to step S284. In step S280, in a case where theoutput image data is input from the image processing circuit 94C, apositive determination is made, and a transition is made to step S282.

In step S282, the output circuit 94D outputs the output image data tothe I/F 56 of the rear stage circuit 90 and then, transitions to stepS284. For example, the I/F 56 outputs the output image data input fromthe output circuit 94D to the CPU 52 and the image processing portion 62through the bus 68.

In step S284, the output circuit 94D determines whether or not the imagedata output processing finish condition described in the firstembodiment is satisfied. In step S284, in a case where the image dataoutput processing finish condition is not satisfied, a negativedetermination is made, and the image data output processing transitionsto step S280. In step S284, in a case where the image data outputprocessing finish condition is satisfied, a positive determination ismade, and the output circuit 94D finishes the image data outputprocessing.

Next, the display control processing executed by the display controlportion of the rear stage circuit 90 will be described with reference toFIG. 9 . For convenience of description, it is assumed that the outputimage data is output to the rear stage circuit 90 from the outputcircuit 94D by executing the image data output processing illustrated inFIG. 8 , and that the output image data is input into the CPU 52 and theimage processing portion 62.

In the display control processing illustrated in FIG. 9 , in step S290,the display control portion determines whether or not the output imagedata is input from the image processing portion 62. In step S290, in acase where the output image data is not input from the image processingportion 62, a negative determination is made, and the display controlprocessing transitions to step S294. In step S290, in a case where theoutput image data is input from the image processing portion 62, apositive determination is made, and the display control processingtransitions to step S292.

In step S292, the display control portion outputs the output image datato the display apparatus as graphics data, and then, the display controlprocessing transitions to step S294. In a case where the output imagedata is output to the display apparatus by executing processing of stepS292, the display apparatus displays the image indicated by the outputimage data. For example, in a case where the main subject image data isincluded in the output image data, the display apparatus displays a mainsubject image indicated by the main subject image data.

While the main subject image is illustrated here, the entirety image maybe displayed in the same manner as the first embodiment, or both of themain subject image and the entirety image may be arranged and displayed.For example, the “entirety image” according to the second embodimentrefers to an image showing the entirety of the subject in a rangefalling within an angle of view imageable by all photodiodes in thepartial region 92B of the photoelectric conversion element 92illustrated in FIG. 24 .

For example, in a case where the single high image quality cut-out imagedata is included in the output image data, the display apparatusarranges and displays the single high image quality cut-out imageindicated by the single high image quality cut-out image data and themain subject image as illustrated in FIG. 24 .

For example, in a case where the enlarged display instruction isreceived by the touch panel 42 and/or the operation portion 54, thedisplay apparatus displays the single high image quality cut-out imagein an enlarged manner as illustrated in FIG. 24 .

While a display example of the single high image quality cut-out imageis illustrated in the example illustrated in FIG. 24 , the displayapparatus displays the single low image quality cut-out image instead ofthe single high image quality cut-out image in a case where the singlelow image quality cut-out image data is included in the output imagedata.

The image quality of the image displayed on the display apparatus issuch that the single high image quality cut-out image has higher imagequality than the single low image quality cut-out image and the mainsubject image.

It is not necessary to display the same image on both of the firstdisplay 40 and the second display 80. For example, the main subjectimage may be displayed on one of the first display 40 and the seconddisplay 80, and the single high image quality cut-out image or thesingle low image quality cut-out image may be displayed on the othertogether with the main subject image. Alternatively, the single highimage quality cut-out image or the single low image quality cut-outimage may be displayed without displaying the main subject image.

Meanwhile, for example, in a case where the plural high image qualitycut-out image data is included in the output image data, the displayapparatus arranges and displays the plurality of high image qualitycut-out images indicated by the plural high image quality cut-out imagedata and the main subject image as illustrated in FIG. 25 . In theexample illustrated in FIG. 25 , a state where two high image qualitycut-out images and the main subject image are arranged and displayed onthe display apparatus is illustrated.

In a case where the enlarged display instruction is received by thetouch panel 42 and/or the operation portion 54, the display apparatusdisplays at least one of the plurality of high image quality cut-outimages in an enlarged manner.

The image quality of the image displayed on the display apparatus issuch that the plurality of high image quality cut-out images have higherimage quality than a plurality of low image quality cut-out images andthe main subject image.

It is not necessary to display the same image on both of the firstdisplay 40 and the second display 80. For example, the main subjectimage may be displayed on one of the first display 40 and the seconddisplay 80, and the plurality of high image quality cut-out images orthe plurality of low image quality cut-out images may be displayed onthe other together with the main subject image. Alternatively, theplurality of high image quality cut-out images or the plurality of lowimage quality cut-out images may be displayed without displaying themain subject image.

In step S294, the display control portion determines whether or not thedisplay control processing finish condition described in the firstembodiment is satisfied. In step S294, in a case where the displaycontrol processing finish condition is not satisfied, a negativedetermination is made, and the display control processing transitions tostep S290. In step S294, in a case where the display control processingfinish condition is satisfied, a positive determination is made, and thedisplay control portion finishes the display control processing.

As described above, in the imaging apparatus 10 according to the secondembodiment, a part of the subject corresponding to the cut-out target ispresent across a plurality of regions, and a plurality of pieces ofregion image data are cut out from the captured image data. The plurallow image quality image data and the plural high image quality cut-outimage data are generated based on the plurality of pieces of cut-outregion image data.

Accordingly, the imaging apparatus 10 according to the second embodimentcan reduce power consumption even in a case where the cut-out target ispresent across a plurality of regions of the subject.

In the example illustrated in FIG. 25 , a range specified by thedesignation address related to the main subject image data overlaps witha range specified by the designation address related to the plural highimage quality cut-out image data.

However, even in a state where the ranges specified by the designationaddresses overlap, the image processing circuit 94C can individuallyacquire the main subject image data and the plural high image qualitycut-out image data since the captured image data is held in the memory96.

The image processing circuit 94C can individually acquire not only theplural high image quality cut-out image data but also the main subjectimage data and the plural low image quality cut-out image data. Inaddition, not only the plural high image quality cut-out image data andthe plural low image quality cut-out image data but also the mainsubject image data and the single high image quality cut-out image dataor the single low image quality cut-out image data may be individuallyacquired. In addition, the entirety image data can be applied instead ofthe main subject image data.

While an example of a form in which two pieces of partial image data arecut out from the captured image data is illustratively described in thesecond embodiment, the technology of the present disclosure is notlimited thereto. For example, three or more pieces of partial image datamay be cut out from the captured image data. In this case, the displaycontrol portion may arrange and display the entirety image and three ormore images based on three or more pieces of partial image data. Amongthree or more images based on three or more pieces of partial imagedata, only a designated image may be displayed in an enlarged manner ordisplayed in a reduced manner.

Third Embodiment

While a method of using the partial image data out of the partial imagedata and the focusing control image data is illustrated in the firstembodiment, a method of using both of the partial image data and thefocusing control image data will be described in the third embodiment.In the third embodiment, the same constituents as constituents describedin the first embodiment will be designated by the same reference signsand will not be described.

The imaging apparatus 10 according to the third embodiment is differentfrom the imaging apparatus 10 according to the first embodiment in thatthe processing circuit 94 executes the image data generation processingillustrated in FIG. 26 instead of the image data generation processingillustrated in FIG. 8 .

In the image data generation processing illustrated in FIG. 26 , first,in step S300, the photoelectric conversion element drive circuit 94Areads out the captured image data from the partial region 92A of thephotoelectric conversion element 92, and then, the image data generationprocessing transitions to step S302.

For example, the captured image data of one frame read out from thepartial region 92A of the photoelectric conversion element 92 byexecuting processing of step S300 is image data in which the horizontallines are thinned out in the vertical direction by skipping two lines ata time as illustrated in FIG. 10 .

In step S302, the AD conversion circuit 94B digitizes and stores thecaptured image data read out by the partial region 92A of thephotoelectric conversion element drive circuit 94A in the memory 96, andthen, the image data generation processing transitions to step S304.

In step S304, the image processing circuit 94C determines whether or notthe enlarged display instruction is received by the touch panel 42and/or the operation portion 54. The enlarged display instructionincludes the designation address described in the first embodiment.

In step S304, in a case where the enlarged display instruction is notreceived by the touch panel 42 and/or the operation portion 54, anegative determination is made, and the image data generation processingtransitions to step S306. In step S304, in a case where the enlargeddisplay instruction is received by the touch panel 42 and/or theoperation portion 54, a positive determination is made, and the imagedata generation processing transitions to step S310.

In steps S306 and S308, the image processing circuit 94C executesprocessing corresponding to processing of steps S106 and S108illustrated in FIG. 7 and then, transitions to step S334.

In step S310, the image processing circuit 94C determines whether or notthe enlarged display instruction received in step S304 is a focusingcontrol enlarged display instruction. The focusing control enlargeddisplay instruction refers to an instruction to cause the displayapparatus to display an image of a location designated in accordancewith the designation address in an enlarged manner and cause theprocessing circuit 94 to generate and output the focusing control imagedata.

In step S310, in a case where the enlarged display instruction receivedin step S304 is not the focusing control enlarged display instruction, anegative determination is made, and the image data generation processingtransitions to step S312. In step S310, in a case where the enlargeddisplay instruction received in step S304 is the focusing controlenlarged display instruction, a positive determination is made, and theimage data generation processing transitions to step S318.

In steps S312 and S314, the image processing circuit 94C executesprocessing corresponding to processing of steps S112 and S114illustrated in FIG. 7 , and then, the image data generation processingtransitions to step S316.

In step S316, the image processing circuit 94C generates and outputsenlarged display image data to the output circuit 94D, and then, theimage processing circuit 94C transitions to step S334. The partial imagedata cut out in step S314 is employed as the enlarged display imagedata. However, the technology of the present disclosure is not limitedthereto. For example, the enlarged display image data may be image dataobtained by performing various types of image processing on the partialimage data cut out in step S314. The enlarged display image datagenerated by executing processing of step S316 is one type of image dataincluded in the output image data.

In steps S318, S320, S322, S324, and S326, the image processing circuit94C executes processing corresponding to processing of steps S118, S120,S122, S124, and S126 illustrated in FIG. 7 . After processing of stepS326 is executed, the image data generation processing transitions tostep S328.

In step S328, the image processing circuit 94C derives a focusingcontrol address based on the designation address acquired in step S324,and then, the image data generation processing transitions to step S330.The focusing control address is an address that can be used forspecifying a part of the captured image data required for the focusingcontrol performed by the CPU 52 from the memory 96 storing the capturedimage data. In step S328, the focusing control address is derived fromthe designation address in accordance with a predetermined derivationtable. However, the technology of the present disclosure is not limitedthereto. For example, the focusing control address may be derived usinga calculation expression that can be used for deriving the same resultas the derivation table.

In step S330, the image processing circuit 94C cuts out the partialimage data from the captured image data of each of the plurality offrames of the captured image data in the memory 96 in accordance withthe focusing control address acquired in step S328, and then, the imagedata generation processing transitions to step S332. Hereinafter, thepartial image data cut out in step S326 will be referred to as “enlargeddisplay partial image data”, and the partial image data cut out in stepS330 will be referred to as “focusing control partial image data”.

For example, the captured image data of the plurality of frames as thecut-out target for the focusing control partial image data is thecaptured image data of all frames currently stored in the memory 96,that is, the captured image data of three frames. However, thetechnology of the present disclosure is not limited thereto. Combineddata obtained by combining the captured image data of one frame or twoframes may be used as the cut-out target.

In step S332, the image processing circuit 94C generates image datacorresponding to the high image quality cut-out image data in step S128illustrated in FIG. 7 as the enlarged display image data, and generatesthe focusing control partial image data based on the enlarged displayimage data. The image processing circuit 94C outputs the generatedenlarged display image data and the focusing control partial image datato the output circuit 94D, and then, the image data generationprocessing transitions to step S334.

For example, combined data obtained by combining the focusing controlpartial image data of three frames cut out in step S330 is employed asthe focusing control image data in the same manner as the exampleillustrated in FIG. 10 . However, the technology of the presentdisclosure is not limited thereto. For example, the focusing controlimage data may be image data obtained by performing various types ofimage processing on the combined data obtained by combining the focusingcontrol partial image data of three frames cut out in step S330. Thefocusing control image data generated by executing processing of stepS332 is one type of image data included in the output image data.

In the example illustrated in FIG. 27 , a width relationship betweenimages of enlarged display image data 200A and focusing control imagedata 352 is shown. For example, as illustrated in FIG. 27 , the imageindicated by the focusing control image data 352 is an image of a rangewider than the image indicated by the enlarged display image data 200A.The “wider range” here means the “predetermined range” described in thefirst embodiment.

For example, in a case where the CPU 52 performs the so-called contrastAF or the phase difference AF in a state where the live view image isdisplayed on the display apparatus in the imaging apparatus 10, a regionof the image indicated by the focusing control image data 352 may beenlarged or reduced in accordance with a blurriness amount. For example,under control of the CPU 52, the image processing circuit 94C adjuststhe focusing control image data 352 such that the region of the imageindicated by the focusing control image data 352 is narrowed as theblurriness amount is decreased. Accordingly, the imaging apparatus 10can reduce power consumption compared to a case where the focusingcontrol image data 352 is constant regardless of the blurriness amount.

In step S334, the image processing circuit 94C determines whether or notthe image data generation processing finish condition described in thefirst embodiment is satisfied. In step S334, in a case where the imagedata generation processing finish condition described in the firstembodiment is not satisfied, a negative determination is made, and theimage data generation processing transitions to step S300. In step S334,in a case where the image data generation processing finish conditiondescribed in the first embodiment is satisfied, a positive determinationis made, and the image processing circuit 94C finishes the image datageneration processing.

Next, the image data output processing executed by the output circuit94D of the processing circuit 94 according to the third embodiment willbe described with reference to FIG. 8 . The image data output processingillustrated in FIG. 8 is performed at the second frame rate by theoutput circuit 94D.

In the image data output processing illustrated in FIG. 8 , first, instep S350, the output circuit 94D determines whether or not the outputimage data is input from the image processing circuit 94C. Image datathat includes at least one of the entirety image data, the enlargeddisplay image data, or the focusing control image data is illustrated asone example of the output image data input into the output circuit 94Dfrom the image processing circuit 94C.

In step S350, in a case where the output image data is not input fromthe image processing circuit 94C, a negative determination is made, anda transition is made to step S354. In step S350, in a case where theoutput image data is input from the image processing circuit 94C, apositive determination is made, and a transition is made to step S351.

In step S351, the output circuit 94D outputs the output image data tothe I/F 56 of the rear stage circuit 90 and then, transitions to stepS354. For example, the I/F 56 outputs the output image data input fromthe output circuit 94D to the CPU 52 and the image processing portion 62through the bus 68.

In step S354, the output circuit 94D determines whether or not the imagedata output processing finish condition described in the firstembodiment is satisfied. In step S354, in a case where the image dataoutput processing finish condition is not satisfied, a negativedetermination is made, and the image data output processing transitionsto step S350. In step S354, in a case where the image data outputprocessing finish condition is satisfied, a positive determination ismade, and the output circuit 94D finishes the image data outputprocessing.

Next, output image data processing executed by the CPU 52 will bedescribed with reference to FIG. 28 .

In the imaging apparatus 10 according to the third embodiment, an outputimage data processing program is stored in the secondary storage portion60, and the CPU 52 reads out the output image data processing programfrom the secondary storage portion 60 and loads the output image dataprocessing program into the primary storage portion 58. The CPU 52executes the output image data processing in accordance with the outputimage data processing program loaded in the primary storage portion 58.

In the output image data processing illustrated in FIG. 28 , in stepS360, the CPU 52 determines whether or not the output image data isinput into the I/F 56 from the processing circuit 94. In step S360, in acase where the output image data is not input into the I/F 56 from theprocessing circuit 94, a negative determination is made, and the outputimage data processing transitions to step S370. In step S360, in a casewhere the output image data is input into the I/F 56 from the processingcircuit 94, a positive determination is made, and the output image dataprocessing transitions to step S362.

In step S362, a determination as to whether or not only the enlargeddisplay image data is included in the output image data is performed. Instep S362, in a case where only the enlarged display image data isincluded in the output image data, a positive determination is made, andthe output image data processing transitions to step S364. In step S362,in a case where the enlarged display image data and the focusing controlimage data are included in the output image data, a negativedetermination is made, and the output image data processing transitionsto step S366.

In step S364, the CPU 52 outputs enlarged display image data to thedisplay control portion, and then, the output image data processingtransitions to step S370. For example, in a case where the enlargeddisplay image data is output to the display control portion, the displaycontrol portion displays an image based on the enlarged display imagedata on the display apparatus as illustrated in FIG. 27 . The imagebased on the enlarged display image data is displayed at the secondframe rate on the display apparatus.

In step S366, the CPU 52 executes processing corresponding to processingof step S364, and then, the output image data processing transitions tostep S368.

In step S368, the CPU 52 performs the focusing control based on thefocusing control image data, and then, the output image data processingtransitions to step S370.

In step S370, the CPU 52 determines whether or not an output image dataprocessing finish condition that is a condition for finishing the outputimage data processing is satisfied. The same condition as the displaycontrol processing finish condition described in the first embodiment isillustrated as one example of the output image data processing finishcondition.

In step S370, in a case where the output image data processing finishcondition is not satisfied, a negative determination is made, and theoutput image data processing transitions to step S360. In step S370, ina case where the output image data processing finish condition issatisfied, a positive determination is made, and the CPU 52 finishes theoutput image data processing.

As described above, in the imaging apparatus 10 according to the thirdembodiment, the focusing control image data is generated, and thegenerated focusing control image data is included in the output imagedata. Accordingly, even in a case where the focusing control isperformed using the focusing control image data, the imaging apparatus10 according to the third embodiment can reduce power consumptioncompared to a case of generating the focusing control image data in therear stage circuit 90.

Fourth Embodiment

The image data generation processing and the image data outputprocessing are illustrated in the first embodiment. In a fourthembodiment, an example of a form in which the processing circuit 94further executes imaging control processing will be described. In thefourth embodiment, the same constituents as constituents described inthe first embodiment will be designated by the same reference signs andwill not be described.

The imaging apparatus 10 according to the fourth embodiment is differentfrom the imaging apparatus 10 according to the first embodiment in thatthe processing circuit 94 can further execute the imaging controlprocessing illustrated in FIG. 29 . In addition, the imaging apparatus10 according to the fourth embodiment is different from the imagingapparatus 10 according to the first embodiment in that the image datageneration processing illustrated in FIG. 30 can be further executed.The imaging control processing illustrated in FIG. 29 and the image datageneration processing illustrated in FIG. 30 are executed at the firstframe rate by the processing circuit 94.

In the imaging control processing illustrated in FIG. 29 , first, instep S400, the processing circuit 94 determines whether or not imaginglocation designation information is received by the touch panel 42and/or the operation portion 54. The imaging location designationinformation refers to information for designating a photodiode to beused for obtaining the partial image data among all photodiodes in thephotoelectric conversion element 92. The imaging location designationinformation includes an imaging location designation address that can beused for specifying a designated position in the photoelectricconversion element 92.

In step S400, in a case where the imaging location designationinformation is not received by the touch panel 42 and/or the operationportion 54, a negative determination is made, and the imaging controlprocessing transitions to step S402. In step S400, in a case where theimaging location designation information is received by the touch panel42 and/or the operation portion 54, a positive determination is made,and the imaging control processing transitions to step S404. A casewhere a positive determination is made in step S400 is one example of a“case where a location corresponding to a part of a subject in aphotoelectric conversion element is designated by designating the partof the subject” according to the embodiment of the technology of thepresent disclosure.

In step S402, the photoelectric conversion element drive circuit 94Aacquires the captured image data by imaging the subject using allphotodiodes in the photoelectric conversion element 92, and then, theimaging control processing transitions to step S414.

In step S404, the photoelectric conversion element drive circuit 94Aacquires the imaging location designation address from the imaginglocation designation information, and then, the imaging controlprocessing transitions to step S406.

In step S406, the photoelectric conversion element drive circuit 94Acauses the photoelectric conversion element 92 to start imaging at animaging location designated in accordance with the imaging locationdesignation address, and then, the imaging control processingtransitions to step S408. The captured image data obtained by imaging byexecuting processing of step S406 is partial image correspondence data.The partial image correspondence data refers to image data correspondingto the partial image data described in the first embodiment. The partialimage correspondence data is stored in the memory 96 in the same manneras the partial image data.

In step S408, the processing circuit 94 acquires the partial imagecorrespondence data from the memory 96, and then, the imaging controlprocessing transitions to step S410.

In step S410, the processing circuit 94 determines whether or not arelease condition for releasing a designation state of the imaginglocation for the photoelectric conversion element 92 is satisfied. Instep S410, in a case where the release condition is not satisfied, anegative determination is made, and the imaging control processingtransitions to step S400. In step S410, in a case where the releasecondition is satisfied, a positive determination is made, and theimaging control processing transitions to step S412.

In step S412, the photoelectric conversion element drive circuit 94Areleases the designation state of the imaging location for thephotoelectric conversion element 92, and then, the imaging controlprocessing transitions to step S414.

In step S414, the processing circuit 94 determines whether or not animaging control processing finish condition that is a condition forfinishing the imaging control processing is satisfied. For example, acondition that an instruction to finish the imaging control processingis received by the touch panel 42 and/or the operation portion 54 isillustrated as the imaging control processing finish condition.

In step S414, in a case where the imaging control processing finishcondition is not satisfied, a negative determination is made, and theimaging control processing transitions to step S400. In step S414, in acase where the imaging control processing finish condition is satisfied,a positive determination is made, and the processing circuit 94 finishesthe imaging control processing.

Next, a case where the image data generation processing according to thefourth embodiment is executed by the processing circuit 94 will bedescribed with reference to FIG. 30 .

In the image data generation processing illustrated in FIG. 30 , in stepS450, the processing circuit 94 determines whether or not image data isacquired from the photoelectric conversion element 92 by thephotoelectric conversion element drive circuit 94A. For convenience ofdescription, the image data acquired from the photoelectric conversionelement 92 is the partial image correspondence data acquired in stepS408 of the imaging control processing or the captured image dataacquired in step S402 of the imaging control processing.

In step S450, in a case where the image data is not acquired from thephotoelectric conversion element 92 by the photoelectric conversionelement drive circuit 94A, a negative determination is made, and theimage data generation processing transitions to step S468. In step S450,in a case where the image data is acquired from the photoelectricconversion element 92 by the photoelectric conversion element drivecircuit 94A, a positive determination is made, and the image datageneration processing transitions to step S452.

In step S452, the processing circuit 94 determines whether or not theimage data acquired from the photoelectric conversion element 92 is thepartial image correspondence data. In step S452, in a case where theimage data acquired from the photoelectric conversion element 92 is notthe partial image correspondence data, that is, in a case where theimage data acquired from the photoelectric conversion element 92 is thecaptured image data, a negative determination is made, and a transitionis made to step S454. In step S452, in a case where the image dataacquired from the photoelectric conversion element 92 is the partialimage correspondence data, a positive determination is made, and theimage data generation processing transitions to step S456. In a casewhere the image data acquired from the photoelectric conversion element92 is the captured image data, the captured image data is stored in thememory 96.

In step S454, the image processing circuit 94C acquires the capturedimage data from the memory 96 and generates and outputs the entiretyimage data described in the first embodiment to the output circuit 94Dbased on the acquired captured image data, and then, the image datageneration processing transitions to step S468.

In step S456, the AD conversion circuit 94B stores the partial imagecorrespondence data in the memory 96, and then, the image datageneration processing transitions to step s458.

In step S458, the image processing circuit 94C determines whether or notthe high image quality cut-out instruction described in the firstembodiment is received by the touch panel 42 and/or the operationportion 54. In step S458, in a case where the high image quality cut-outinstruction is not received by the touch panel 42 and/or the operationportion 54, a negative determination is made, and the image datageneration processing transitions to step S460. In step S458, in a casewhere the high image quality cut-out instruction is received by thetouch panel 42 and/or the operation portion 54, a positive determinationis made, and the image data generation processing transitions to stepS462.

In step S460, the image processing circuit 94C acquires the partialimage correspondence data of a single frame from the memory 96. Theimage processing circuit 94C generates and outputs low image qualitycut-out image correspondence data to the output circuit 94D based on theacquired partial image correspondence data of the single frame, andthen, the image data generation processing transitions to step S468.

For example, the partial image correspondence data of the single framerefers to the partial image correspondence data least recently stored inthe memory 96. However, the technology of the present disclosure is notlimited thereto. For example, the partial image correspondence data ofthe single frame may be any partial image correspondence data in a casewhere the partial image correspondence data of a plurality of frames isstored in the memory 96.

The low image quality cut-out image correspondence data refers to imagedata corresponding to the low image quality cut-out image data describedin the first embodiment.

In step S462, the image processing circuit 94C determines whether or notthe captured image data of a plurality of frames is stored in the memory96. In step S462, three frames are employed as one example of theplurality of frames. In step S462, in a case where the captured imagedata of the plurality of frames is not stored in the memory 96, anegative determination is made, and the image data generation processingtransitions to step S464. In step S462, in a case where the capturedimage data of the plurality of frames is stored in the memory 96, apositive determination is made, and the image data generation processingtransitions to step S466.

In step S464, the image processing circuit 94C executes processingcorresponding to processing of step S454, and then, the image datageneration processing transitions to step S462.

In step S466, the image processing circuit 94C acquires the partialimage correspondence data of the plurality of frames from the memory 96.The image processing circuit 94C generates and outputs high imagequality cut-out image correspondence data to the output circuit 94Dbased on the acquired partial image correspondence data of the pluralityof frames, and then, the image data generation processing transitions tostep S468.

The high image quality cut-out image correspondence data refers to imagedata corresponding to the high image quality cut-out image datadescribed in the first embodiment. For example, combined data obtainedby combining the partial image correspondence data of three frames isemployed as the high image quality cut-out image correspondence data inthe same manner as the example illustrated in FIG. 10 . However, thetechnology of the present disclosure is not limited thereto. Forexample, the high image quality cut-out image correspondence data may beimage data obtained by performing various types of image processing onthe combined data obtained by combining the partial image data of threeframes.

The low image quality cut-out image data generated by executingprocessing of step S460 and the high image quality cut-out image datagenerated by executing processing of step S466 are one example of “imagedata based on partial image correspondence data” according to theembodiment of the technology of the present disclosure.

In step S468, the image processing circuit 94C determines whether or notthe image data generation processing finish condition described in thefirst embodiment is satisfied. In step S468, in a case where the imagedata generation processing finish condition is not satisfied, a negativedetermination is made, and the image data generation processingtransitions to step S450. In step S468, in a case where the image datageneration processing finish condition is satisfied, a positivedetermination is made, and the image processing circuit 94C finishes theimage data generation processing.

Hereinafter, for convenience of description, the low image qualitycut-out image correspondence data and the high image quality cut-outimage correspondence data will be referred to as “cut-out imagecorrespondence data” unless otherwise necessary to distinguishtherebetween for description.

Next, the image data output processing executed by the output circuit94D of the processing circuit 94 according to the fourth embodiment willbe described with reference to FIG. 8 . The image data output processingillustrated in FIG. 8 according to the fourth embodiment is performed atthe second frame rate by the output circuit 94D.

In the image data output processing illustrated in FIG. 8 according tothe fourth embodiment, first, in step S490, the output circuit 94Ddetermines whether or not the cut-out image correspondence data is inputfrom the image processing circuit 94C. In step S490, in a case where thecut-out image correspondence data is not input from the image processingcircuit 94C, a negative determination is made, and a transition is madeto step S494. In step S490, in a case where the cut-out imagecorrespondence data is input from the image processing circuit 94C, apositive determination is made, and a transition is made to step S492.

In step S492, the output circuit 94D outputs the cut-out imagecorrespondence data to the I/F 56 of the rear stage circuit 90 and then,transitions to step S494. For example, the OF 56 outputs the cut-outimage correspondence data input from the output circuit 94D to the CPU52 and the image processing portion 62 through the bus 68.

In step S494, the output circuit 94D determines whether or not the imagedata output processing finish condition is satisfied. In step S494, in acase where the image data output processing finish condition is notsatisfied, a negative determination is made, and the image data outputprocessing transitions to step S490. In step S494, in a case where theimage data output processing finish condition is satisfied, a positivedetermination is made, and the output circuit 94D finishes the imagedata output processing.

Next, the display control processing executed by the display controlportion of the rear stage circuit 90 according to the fourth embodimentwill be described with reference to FIG. 9 . For convenience ofdescription, it is assumed that the cut-out image correspondence data isoutput to the rear stage circuit 90 from the output circuit 94D byexecuting the image data output processing illustrated in FIG. 8 , andthat the cut-out image correspondence data is input into the CPU 52 andthe image processing portion 62. In addition, an image indicated by thehigh image quality cut-out image correspondence data and an imageindicated by the low image quality cut-out image correspondence datawill be referred to as a “cut-out correspondence image” unless otherwisenecessary to distinguish therebetween for description.

In the display control processing illustrated in FIG. 9 , in step S500,the display control portion determines whether or not the cut-out imagecorrespondence data is input from the image processing portion 62. Instep S500, in a case where the cut-out image correspondence data is notinput from the image processing portion 62, a negative determination ismade, and the display control processing transitions to step S504. Instep S500, in a case where the cut-out image correspondence data isinput from the image processing portion 62, a positive determination ismade, and the display control processing transitions to step S502.

In step S502, the display control portion outputs the cut-out imagecorrespondence data to the display apparatus as graphics data, and then,the display control processing transitions to step S504. In a case wherethe cut-out image correspondence data is output to the display apparatusby executing processing of step S502, the display apparatus displays thecut-out correspondence image indicated by the cut-out imagecorrespondence data.

The cut-out correspondence image is any of a high image quality cut-outcorrespondence image and a low image quality cut-out correspondenceimage. For example, the high image quality cut-out correspondence imageis an image corresponding to the image based on the combined dataobtained by combining the captured image data of three frames in thesame manner as the example illustrated in FIG. 10 . Meanwhile, the lowimage quality cut-out correspondence image is a thinned image based onthe captured image data of one frame. Thus, the image quality of theimage displayed on the display apparatus is such that the high imagequality cut-out correspondence image has higher image quality than thelow image quality cut-out correspondence image.

In step S504, the display control portion determines whether or not thedisplay control processing finish condition is satisfied. In step S504,in a case where the display control processing finish condition is notsatisfied, a negative determination is made, and the display controlprocessing transitions to step S500. In step S504, in a case where thedisplay control processing finish condition is satisfied, a positivedetermination is made, and the display control portion finishes thedisplay control processing.

For example, by executing the imaging control processing illustrated inFIG. 29 , the image data generation processing illustrated in FIG. 30 ,the image data output processing illustrated in FIG. 8 , and the displaycontrol processing illustrated in FIG. 9 , the live view image isdisplayed on the display apparatus under control of the display controlportion. In this case, the “live view image” refers to an image in alive view format based on the cut-out image correspondence data. In theimaging apparatus 10 according to the fourth embodiment, the live viewimage is displayed at the second frame rate.

As described above, in the imaging apparatus 10 according to the fourthembodiment, the processing circuit 94 generates the partial imagecorrespondence data by causing the photoelectric conversion element 92to perform imaging at the imaging location designated in accordance withthe imaging location designation address, and stores the partial imagecorrespondence data in the memory 96. The processing circuit 94 acquiresthe partial image correspondence data from the memory 96. The outputcircuit 94D outputs the cut-out image correspondence data to the rearstage circuit 90.

Accordingly, the imaging apparatus 10 according to the fourth embodimentcan reduce power consumption compared to a case of imaging the subjectusing all regions of the photoelectric conversion element 92 at alltimes.

While a case where compressed image data is used for displaying the liveview image is described in each embodiment above, the technology of thepresent disclosure is not limited thereto. For example, by the CPU 52,the compressed image data may be stored in the secondary storage portion60 in the rear stage circuit 90 or may be output to the outside of theimaging apparatus 10 through the external I/F 63.

While the processing circuit 94 implemented by the ASIC is illustratedin each embodiment above, the image data generation processing, theimage data output processing, and the imaging control processing may beimplemented by a computer using a software configuration.

In this case, for example, as illustrated in FIG. 31 , a program 600 forcausing a computer 20A incorporated in the imaging element 20 to executethe image data generation processing, the image data output processing,and the imaging control processing is stored in a storage medium 700.The computer 20A comprises a CPU 20A1, a ROM 20A2, and a RAM 20A3. Theprogram 600 of the storage medium 700 is installed on the computer 20A,and the CPU 20A1 of the computer 20A executes the image data generationprocessing, the image data output processing, and the imaging controlprocessing in accordance with the program 600. A single CPU isillustrated as the CPU 20A1. However, the technology of the presentdisclosure is not limited thereto, and a plurality of CPUs may beemployed instead of the CPU 20A1. That is, the image data generationprocessing, the image data output processing, and/or the imaging controlprocessing may be executed by one processor or a plurality of physicallyseparated processors.

Any portable storage medium such as a solid state drive (SSD) or auniversal serial bus (USB) is illustrated as one example of the storagemedium 700.

Alternatively, the program 600 may be stored in a storage portion ofanother computer, a server apparatus, or the like connected to thecomputer 20A through a communication network (not illustrated), and theprogram 600 may be downloaded in accordance with a request from theimaging apparatus 10 or the like. In this case, the downloaded program600 is executed by the computer 20A.

The computer 20A may be disposed outside the imaging element 20. In thiscase, the computer 20A may control the processing circuit 94 inaccordance with the program 600.

Various processors illustrated below can be used as a hardware resourcefor executing various types of processing described in each embodimentabove. As various types of processing described in each embodimentabove, the image data generation processing, the image data outputprocessing, the display control processing, the single region cut-outprocessing, the plural region cut-out processing, the output image dataprocessing, and the imaging control processing are illustrated. Forexample, as described above, a CPU that is a general-purpose processorfunctioning as a hardware resource for executing various types ofprocessing according to the embodiment of the technology of the presentdisclosure by executing software, that is, the program, is illustratedas a processor. In addition, a dedicated electric circuit such as anFPGA, a PLD, or an ASIC that is a processor having a circuitconfiguration dedicatedly designed to execute a specific type ofprocessing is illustrated as a processor. A memory is incorporated in orconnected to any of the processors, and any of the processors executesvarious types of processing using the memory.

The hardware resource for executing various types of processingaccording to the embodiment of the technology of the present disclosuremay be configured with one of those various processors or may beconfigured with a combination of two or more processors of the same typeor different types (for example, a combination of a plurality of FPGAsor a combination of a CPU and an FPGA). Alternatively, the hardwareresource for executing various types of processing according to theembodiment of the technology of the present disclosure may be oneprocessor.

As an example of a configuration with one processor, first, asrepresented by a computer such as a client and a server, a form in whichone processor is configured with a combination of one or more CPUs andsoftware and this processor functions as the hardware resource forexecuting various types of processing according to the embodiment of thetechnology of the present disclosure is available. Second, asrepresented by a system-on-a-chip (SoC) or the like, a form of using aprocessor that implements, by one IC chip, a function of the entiresystem including a plurality of hardware resources for executing varioustypes of processing according to the embodiment of the technology of thepresent disclosure is available. Accordingly, various types ofprocessing according to the embodiment of the technology of the presentdisclosure are implemented using one or more of above various processorsas a hardware resource.

Furthermore, as a hardware structure of those various processors, morespecifically, an electric circuit in which circuit elements such assemiconductor elements are combined can be used.

While an interchangeable lens camera is illustrated as the imagingapparatus 10 in each embodiment above, the technology of the presentdisclosure is not limited thereto. For example, the technology of thepresent disclosure may be applied to a smart device 900 illustrated inFIG. 32 . For example, the smart device 900 illustrated in FIG. 32 isone example of the imaging apparatus according to the embodiment of thetechnology of the present disclosure. The imaging element 20 describedin the embodiments is mounted on the smart device 900. Even with thesmart device 900 configured in such a manner, the same action and effectas the imaging apparatus 10 described in each embodiment above areachieved. The technology of the present disclosure can be applied to notonly the smart device 900 but also a personal computer or a wearableterminal apparatus.

While the first display 40 and the second display 80 are illustrated asthe display apparatus in each embodiment above, the technology of thepresent disclosure is not limited thereto. For example, a separatedisplay that is retrofit into the imaging apparatus main body 12 may beused as the “display portion” according to the embodiment of thetechnology of the present disclosure.

The image data generation processing, the image data output processing,the display control processing, the output image data processing, andthe imaging control processing described in each embodiment above aremerely one example. Accordingly, unnecessary steps may be removed, newsteps may be added, or a processing order may be changed withoutdeparting from a gist of the technology of the present disclosure.

Above described contents and illustrated contents are detaileddescription for parts according to the embodiment of the technology ofthe present disclosure and are merely one example of the technology ofthe present disclosure. For example, description related to the aboveconfigurations, functions, actions, and effects is description relatedto one example of configurations, functions, actions, and effects of theparts according to the embodiment of the technology of the presentdisclosure. Thus, unnecessary parts may be removed, new elements may beadded, or parts may be replaced in the above described contents and theillustrated contents without departing from the gist of the technologyof the present disclosure. In addition, particularly, descriptionrelated to common technical knowledge or the like that does not need tobe described in terms of embodying the technology of the presentdisclosure is omitted in the above described contents and theillustrated contents in order to avoid complication and facilitateunderstanding of the parts according to the embodiment of the technologyof the present disclosure.

In the present specification, “A and/or B” has the same meaning as “atleast one of A or B”. This means that “A and/or B” may be only A, may beonly B, or may be a combination of A and B. In addition, in the presentspecification, the same approach as “A and/or B” is applied to a casewhere three or more matters are represented by connecting the matterswith “and/or”.

All documents, patent applications, and technical standards disclosed inthe present specification are incorporated in the present specificationby reference to the same extent as in a case where each of thedocuments, patent applications, technical standards is specifically andindividually indicated to be incorporated by reference.

What is claimed is:
 1. An imaging element comprising: a memory thatstores captured image data obtained by imaging a subject at a firstframe rate, the memory being incorporated in the imaging element; animage processing circuit that performs processing on the captured imagedata that is stored in the memory to generate output image data, theimage processing circuit being incorporated in the imaging element; andan output circuit that outputs all output image data obtained byperforming the processing on the captured image data to an exterior ofthe imaging element at a second frame rate, the output circuit beingincorporated in the imaging element, wherein the processing performed bythe image processing circuit comprises cut-out processing with respectto one frame of the captured image data, the cut-out processingincluding cutting out partial image data indicating an image of a partof the subject in the captured image data from a designated address inthe memory, the output image data includes image data based on thepartial image data that is cut out from the captured image data as aresult of performing the cut-out processing by the image processingcircuit, the first frame rate is a frame rate higher than the secondframe rate, and wherein the image processing circuit performs theprocessing at the first frame rate.
 2. The imaging element according toclaim 1, wherein the cut-out processing is processing of cutting out thepartial image data from the captured image data stored in the memory byrandom access to the memory.
 3. The imaging element according to claim1, wherein the memory is capable of storing the captured image data of aplurality of frames, the cut-out processing includes processing ofcutting out the partial image data of the plurality of frames from thecaptured image data stored in the memory, and the output portionoutputs, as the output image data, image data that is obtained bycombining the partial image data of the plurality of frames cut out as aresult of performing the cut-out processing by the image processingcircuit.
 4. The imaging element according to claim 3, wherein thepartial image data is at least one of a plurality of pieces of dividedregion image data corresponding to a plurality of divided regionsobtained by dividing the part.
 5. The imaging element according to claim4, wherein the plurality of pieces of divided region image data have arelationship in which mutually different pixels are thinned out in unitsof lines.
 6. The imaging element according to claim 4, wherein theoutput image data includes combined data obtained by combining theplurality of pieces of divided region image data.
 7. The imaging elementaccording to claim 6, wherein the output circuit selectively outputs, asthe partial image data, divided region image data of a part of theplurality of pieces of divided region image data or the combined data inaccordance with a provided condition.
 8. The imaging element accordingto claim 1, wherein the subject is imaged by a photoelectric conversionelement, in a case in which an area of the photoelectric conversionelement corresponding to the part is designated by designating the partof the subject, the image processing circuit causes the area of thephotoelectric conversion element to perform imaging, and the imageprocessing circuit acquires partial image correspondence datacorresponding to the partial image data at the first frame rate byperforming imaging by the area of the photoelectric conversion element,and outputs image data based on the acquired partial imagecorrespondence data to the exterior of the imaging element at the secondframe rate.
 9. The imaging element according to claim 1, wherein thepart includes a plurality of regions in the subject, and the partialimage data is a plurality of pieces of region image data indicating theplurality of regions in the captured image data.
 10. The imaging elementaccording to claim 1, wherein the output image data further includeswide range image data that indicates an image of a range of the subjectwider than the part of the subject, and resolution of the wide rangeimage data is lower than resolution of the partial image data.
 11. Theimaging element according to claim 10, wherein the wide range image datais image data indicating an image of an entirety of the subject.
 12. Theimaging element according to claim 1, wherein the output image datafurther includes focusing control image data indicating an image of arange that is a range of the subject wider than the part of the subjectand that is a predetermined range as a range required for a focusingcontrol of an imaging apparatus including the imaging element.
 13. Theimaging element according to claim 1, wherein the imaging element is astacked imaging element that includes a photoelectric conversionelement, and in which the photoelectric conversion element is stacked onthe memory.
 14. An imaging apparatus comprising: the imaging elementaccording to claim 1; and a display processor that is configured toperform a control for displaying an image based on the output image dataoutput by the image processing circuit on a display in an enlargedmanner.
 15. An image data processing method of an imaging element inwhich a memory, an image processing circuit, and an output circuit areincorporated, the image data processing method comprising: storing, bythe memory, captured image data obtained by imaging a subject at a firstframe rate; performing, by the image processing circuit, processing onthe captured image data that is stored in the memory to generate outputimage data; and outputting, by the output circuit, all output image dataobtained by performing the processing on the captured image data to anexterior of the imaging element at a second frame rate, wherein theprocessing performed, by the image processing circuit comprises cut-outprocessing with respect to one frame of the captured image data, thecut-out processing including cutting out partial image data indicatingan image of a part of the subject in the captured image data from adesignated address in the memory, wherein the output image data includesimage data based on the partial image data that is cut out from thecaptured image data as a result of performing the cut-out processing bythe image processing circuit, the first frame rate is a frame ratehigher than the second frame rate, and wherein the image processingcircuit performs the processing at the first frame rate.
 16. Anon-transitory storage medium storing a program that causes a computerto function as an image processing circuit and an output portionincluded in an imaging element, and to perform an image data processing,the imaging element incorporating a memory, the image processingcircuit, and the output circuit, the image data processing comprising:storing, by the memory, captured image data obtained by imaging asubject at a first frame rate; performing, by the image processingcircuit, processing on the captured image data that is stored in thememory to generate output image data; and outputting, by the outputcircuit, all output image data obtained by performing the processing onthe captured image data to an exterior of the imaging element at asecond frame rate, wherein the processing performed, by the imageprocessing circuit comprises cut-out processing with respect to oneframe of the captured image data, the cut-out processing includingcutting out partial image data indicating an image of a part of thesubject in the captured image data from a designated address in thememory, wherein the output image data includes image data based on thepartial image data that is cut out from the captured image data as aresult of performing the cut-out processing by the image processingcircuit, the first frame rate is a frame rate higher than the secondframe rate, and wherein the image processing circuit performs theprocessing at the first frame rate.
 17. The imaging element according toclaim 1, wherein all of the captured image data obtained by the imagingis once stored in the memory and then output from the output circuit viathe image processing circuit.
 18. The imaging element according to claim1, further comprising a photoelectric conversion element that generatesthe captured image data by imaging the subject, wherein an outputterminal of the photoelectric conversion element is connected to thememory, an output terminal of the memory is connected to the imageprocessing circuit, and an output terminal of the image processingcircuit is connected to the output circuit.